Homomorphic Encryption 101

I was recently exploring methods for improved privacy using various encryption schemes and stumbled upon Homomorphic Encryption that has a huge potential  in that area. I do feel that it has higher barrier to entry considering the complexity and level of maturity it has today. If you’re looking for learning resources/libraries to get started on it take a look at Git repo that I have created for purpose of sharing resources around Homomorphic Encryption.

At a very high level Holomorphic Encryption allows you to perform basic matematical computations (+,-,x,/) on encrypted data (cipher text) without need to have un-encrypted data (plaintext). This ability to perform operations on encrypted data has many high impact use cases.

Just to give you an idea lets say you like to leverage a service hosted by a cloud provider but you don’t want to reveal the data to could provider without it being encrypted.  The biggest challenge today is without access to actual data (in decrypted form) there is very limited useful operations that can be performed on it. However, with Homomorphic Encryption  cloud provider can take your data which in encrypted form then process it without decrypting it and then gives you back the result which is also encrypted. At no point  your data is revealed to the cloud provider in decrypted form.

The biggest beneficiary of this type of encryption is privacy. So, at this point you may ask well if this is so useful why it hasn’t been adopted/used commercially on a wider scale? Well, the short answer is that Homomorphic Encryption is still in its infancy. This article call out some of the challenges that you may want to look into. In short its still being actively worked upon and organization like NIST are working towards its standardization.

Finally, let me leave you with a simple example using Python Paillier library. I will use set of numbers and encrypt them using private key and then use the library (think of it as cloud provider though I’m running everything on my laptop using docker container) perform the mathematical operations (+,-.*,/) on the numbers while they are encrypted. Only thing the library needs is the public key. After the operations  are done the results are provided back which are also encrypted. At the end you decrypt the results using your private key. In short at no point library has access to your un-encrypted data. There is another library that is quite useful for trying Homomorphic Encryption called  SEAL (Simple Encrypted Arithmetic Library) by Microsoft which I also experimented with but going to cover it in a separate post.

As I mentioned earlier, I’m using Docker container image that I created to package the Python Paillier library which is available on DockerHub.


#Launch Docker container and remove it automatically afterwords.
docker run --rm -it rbinrais/python-paillier:1.2.2 bash


xxxxxx@xxxxxxxxxxx:/# python3
Python 3.5.2 (default, Nov 17 2016, 17:05:23) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.

#Import Library
from phe import paillier

#Generate a Private/Public Key Pair
public_key, private_key = paillier.generate_paillier_keypair() 

#Define Numbers 
secret_number_list = [12, 2.89763, -4.6e-12] 

#Encrypt Numbers (Using Public Key)
encrypted_number_list = [public_key.encrypt(x) for x in secret_number_list]

#List Encrypted Numbers
encrypted_number_list
[<phe.paillier.EncryptedNumber object at 0x7efd57c0f630>, <phe.paillier.EncryptedNumber object at 0x7efd57c16358>, <phe.paillier.EncryptedNumber object at 0x7efd553229b0>]

#Decrypt Numbers (Using Private Key)
[private_key.decrypt(x) for x in encrypted_number_list]
[12, 2.89763, -4.6e-12]

#Perform Mathematical Operations 
a, b, c = encrypted_number_list
a_plus_10 = a + 10
a_mins_b = a - b 
b_times_4_7 = b * 4.7 
c_div_33 = c / 33

#Display Encrypted Results 
a_plus_10
<phe.paillier.EncryptedNumber object at 0x7efd57c0f668>

a_mins_b
<phe.paillier.EncryptedNumber object at 0x7efd57c0f5c0>

b_times_4_7
<phe.paillier.EncryptedNumber object at 0x7efd55d03240>

c_div_33 
<phe.paillier.EncryptedNumber object at 0x7efd55d03978>

#Decrypt Results using Private Key
private_key.decrypt(a_plus_10)
22

private_key.decrypt(a_mins_b) 
9.10237

private_key.decrypt(b_times_4_7)
13.618861

private_key.decrypt(c_div_33)   
-1.393939393939394e-13

Creating Developer’s Docker Linux Virtual Machine on Azure


For an upcoming developer event on Docker I had to create handful of Linux Ubuntu virtual machines on Azure with Docker and few additional software installed on it.

I looked into couple of ways to to do that on Azure in a consistent fashion. The first option was to use DevTest labs and use artifacts. Another option is to use Custom Extensions. There are other options including creating your own base virtual machine image with all the software installed and then upload it on Azure. I picked custom extension approach mainly because its the simplest approach and I knew the software that I needed to install won’t take more than ~5 minutes on average. It also has a reasonable tradeoff (speed of deployment versus managing your own virtual machine image etc.)

Anyways, the actual process to leverage custom extensions are rather straightforward. Create the scripts. Create the scripts and then call them in your ARM Template (which is a JSON file).

Here is the complete list of software. I choose to use Ubuntu 16.04 LTS Azure Virtual Machine image so that wasn’t needed to be installed.

  • Docker (Engine & Client)
  • Git
  • Nodejs
  • Dotnetcore
  • Yeoman
  • Bower
  • Azure Command Line Interface (CLI)

The approach I took was to create a single script file  for each one of them to keep things simple and clean.

2017-07-22_14-13-45

Once done with the scripts all I need to do is reference/call the install.sh script from the custom extension. Take a look at it on at line 211 in JSON.

If you like to look at the code artifacts I have made them available at Git repo. You can also simply try out creating a virtual machine by single clicking “Deploy on Azure” button. You do need an active Azure subscription before you can deploy virtual machine on Azure.

2017-07-22_14-29-56

Event Announcement “Blockchain 101 – Introduction for Developers”


Some of you may already be aware that I host NYC MS Cloud User Group technology meet up every month at Microsoft Manhattan campus.This month, I will be hosting/presenting alongside with my colleague Cale Teeteron blockchain.I did a similar session earlier this year in January and turnout was great and based on feedback doing another session in July.

Here is the brief agenda:

  • Learn basics of blockchain. What exactly is a block? How blocks are created? What are transactions?
  • Understand what is a transaction and role of mining.
  • Learn what are smart contracts and how to write them in solidity.
  • Demos (mostly based on Ethereum but will talk about other chains too as its important to understand the overall landscape)

Blockchain 101 – Introduction for Developers

Monday, Jul 31, 2017, 6:30 PM

Location details are available to members only.

62 Members Attending

Hello everyone!Excited to announce first session for summer and its on blockchain! (again considering the demand)Here is what Gartner predicts about blockchain:·  By 2022, at least one innovative business built on blockchain technology will be worth $10 billion.·  By 2030, 30% of the global customer base will be made up of things, and those thi…

Check out this Meetup →

 

DevOps with Containers

Recently I did a video series for Microsoft Channel9 on DevOps with Containers (thanks to Lex Thomas and Chris Caldwell for recording these). The idea was simple- show and tell how container technology can help in improving the DevOps experience.

Its a ~2-hour long recording (divided into three parts for easy viewing) covers topics including containerization of applications, continuous integration and deployment of containerized applications using Visual Studio Team System, Azure Container Services, Docker Swarm, DC/OS and monitoring containers using Operations Management Suite and 3rd party tools.

Here is the break down of each session. If you’re interested in looking at the sample application that I have deployed in the last session (asp net core web app and ape) its available on my Git repo.

Part 1 – Getting Started with Containers

In the first part focus is to introduce the basic concepts of container and the process of application containerization. I did target Windows Containers in this part though later parts do show how to leverage multi-container application based on ASP.NET Core using Linux container. If you wanted to try Windows Containers I have provided this link that will allow you to automatically provision Windows Server 2016 Virtual Machine with containers support (including docker-compose). Also, the Azure ARM Template that actually provision the virtual machine is available here.

  • [2:01] What is a Container and how can it benefit organizations?
  • [5:20DEMO: Windows Containers 101- Basics and Overview
  • [9:33DEMO: How to create a Container on Nano Server
  • [15:39DEMO: Windows Server Core and Containers
  • [19:36DEMO: How to containerize legacy ASP.NET 4.5 application
  • [43:48DEMO: Running  Microsoft SQL Server Express inside Container

Part 2 – Building CI/CD pipeline with VSTS and Azure Container Service

The second part focus on building a Continuous Integration (CI) and Continuous Deployment (CD) pipeline for multi container application using Visual Studio Team System (VSTS) with deployment target of Azure Container Service (ACS) hosting DC/OS and Docker Swarm.

I developed a sample application that represents a canonical web app and ape (in this case I used ASP.NET Core 1.1 but really can be NodeJS, Python , Java etc.). Then demos show workflow that starts by submitting code along with Dockerfile and docker-compose that actually will be used by VSTS build to create a new container image every time build is run {container name:buildnumber} format. Containers are hosted in Azure Container Registry which is a private DTR (docker trusted registry). After container image is ready the continuous deployment happens and VSTS kicks the release which targets both DC/OS and Docker Swarm that are actually hosted on Azure Container Service (ACS).

  • [2:54] The Big Picture – Making DevOps successful
  • [6:34DEMO: Building a Continuous Integration and Continuous Deployment system with Azure Container Service and Visual Studio Team System
    • Multi-Container Application | ASP.NET Core
    • Container Images Storage | Azure Private Docker Registry
    • Build & Release Deployment | Visual Studio Team System

Part 3 (Final) – Monitoring and Analytics

This is the final part which focuses on doing Monitoring and Analytics of container applications running on Azure Container Service. Microsoft Operations Management Suite (OMS) is the primarily service used in the demos but I did mention 3rd party services that are supported on Azure Container Service and provide monitoring, analytics and debugging functionality

  • [3:20] Does Orchestration = Containers?
  • [5:40] DEMO: Monitoring and Analytics

Final Thoughts

Containers are massively useful technology for both Green Field and Brown field based application development. Also, organizations today have various level of maturity level when it comes to DevOps and containers provide them with great option to enable DevOps in an effective way. Off course there are considerations like learning curve, lack of proven practices and reference architectures compare to traditional technologies. However, this is going to be lesser concern as with time the knowledge gap going to be filled and reference architectures will emerge.

Finally, you should also broaden your design choices to include combination of containers with server less computing (e.g. Azure Function which actually runs inside a container itself!). This is particularly interning option when your service is mainly state less. This is something I would like to cover in future blog post.