Java SDK for Hyperledger Fabric 1.0 (IBM Blockchain) — Setting Up the Environment

Hello,

IBM Blockchain is based the Linux Foundation’s open source Hyperledger Fabric project, although not all implementations of Hyperledger Fabric version 1.0 ALPHA are also IBM Blockchain. At the time of this writing (May 2017), the Hyperledger Fabric project is at version 1.0-alpha, targeting a launch for version 1.0 proper sometime in mid to late June.

Update 2018-05-03 fixed a Powershell command typo caused by WordPress converting to HTML special characters.

What we intend to build

After a quick overview of what is this blockchain technology, I will step through the building of the recommended Linux Virtual machine on a Windows host, start Docker microservices running Hyperledger Fabric on that virtual image, build the fabric-sdk-java which can be found at https://github.com/hyperledger/fabric-sdk-java and run the java unit tests. Next, we will run the end to end Java integration tests. Finally, we will build the HTML documentation with Sphinx.

If you are faint-of-heart but nevertheless curious, complete text file listings of the operations will be provided below. If you want you, can simply go over those files first if building all this seems a bit too daunting.

Assumptions

If you intent to replicate, here are my assumptions:

You know how to set a new Windows system variable through the Control Panel > System. You have administrator privileges on your Windows box.

You are not daunted by command prompts.

You like to edit configuration files in vi on Linux.

You looked at the docs: https://hyperledger-fabric.readthedocs.io/en/latest/index.html

Ramp-up references and cool videos

There is a free self-paced course offered by IBM at  https://developer.ibm.com/courses/all-courses/blockchain-for-developers/ which is helpful to understand the basics of blockchain and Hyperledger. You can follow it by logging in using a (free) IBM Bluemix account. But mostly this 10 hour course refers to Youtube videos, so in short, here below is just the good stuff. Some of the videos have dodgy titles which explains why they are so badly indexed on Google.

Use cases (9 minutes)

What is a fabric? (5 minutes)

Participants in a blockchain (3 minutes)

How Apps Use the Ledger (4 minutes)

Transactions (8 minutes, 5 stars!)

Integrating with existing systems (3 minutes)

Hyperledger Fabric Architecture (9 minutes)

Bluemix demo:

New terms you might want to remember:

• Practical Byzantine Fault Tolerance (PBFT) concensus protocol
• Ecerts (Enrollment Certificates)
• Tcerts (Transaction Certificates)
• Treshhold Cryptography
• Blockchain event

Overall Steps we will follow:

• Enable virtualization in the BIOS
• Install Chocolatey with Powershell
• Install Vagrant and Virtualbox with Powershell
• Install Cygwin with Powershell
• Install GnuPg on Cygwin using Powershell
• Install OpenSSH on Cygwin using Powershell
• Install Rsync on Cygwin using Powershell (this step might be optional)
• Install Ncurses on Cygwin using Powershell (this step might be optional)
• Install Git on Cygwin using Powershell
• Install Python on Cygwin using Powershell
• Install make on Cygwin using Powershell
• Install Sphinx on Cygwin using Powershell
• Install the Go programming language on Windows
• Create a Go working folder and set up a GOPATH Windows system variable
• Test Go in Cygwin
• In Cygwin, clone Hyperledger Fabric and the Fabric Certificate Authority with Git, clone the Java SDK, tweak the Vagrant config file and build the Linux Ubuntu virtual image
• Check the folders shared between Windows and the Linux virtual image
• Build Fabric and Fabric-ca inside Docker microservices running on the Linux virtual image
• Building the fabric-sdk-java and running the unit tests
• Running the end-to-end Java integration tests
• Building the Fabric HTML documentation with Sphinx

Enable virtualization in the BIOS

Required for Virtualbox.

Boot the PC go into the BIOS. Set the Virtualization and VT-d to enabled. I am running a Windows 7 box.

Install Chocolatey with Powershell

Chocolatey is a series of Powershell automation scripts which will help us install Vagrant, Virtualbox and Cygwin. Reference: chocolatey.org

Open a Powershell prompt AS AN ADMINISTRATOR. Commands:

Get-ExecutionPolicy

If the response is « Restricted » as here below :

Then do both these commands (the Chocolatey site seems to say either is required, but I found both are):

Set-ExecutionPolicy AllSigned

Set-ExecutionPolicy Bypass

(and accept with Y for each)

Install Chocolatey with:

iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))

(and accept with Y)

Close the Powershell prompt.

Install Vagrant et Virtualbox with Powershell

Open a new Powershell prompt AS AN ADMINISTRATOR. Do command:

cinst vagrant virtualbox

(and accept with Y)

Here a Windows reboot is required..

Install Cygwin with Powershell

Open a new Powershell prompt AS AN ADMINISTRATOR. Do command:

cinst cyg-get

(and accept with Y)

Close the Powershell prompt.

Cygwin has now been installed under C:\tools but we will need to add a couple of features to it. Admittedly, a few of these commands could have been lumped together. They were the result of much hit and miss.

Install various useful development tools in Cygwin using Powershell

Open a new Powershell prompt AS AN ADMINISTRATOR. Do command:

cyg-get gnupg openssh ncurses rsync git make python python-sphinx

Install the Go programming language on Windows

Google’s Go programming language will be required BOTH on Windows and in the Linux virtual image. We have to worry about the former, the latter is handled for us. FYI, Go is also the programming language used to program the microservice container tool Docker.

Download Go from https://golang.org/dl/

Install the MSI:

Accept all suggested values.

Add C:\Go\bin to the Windows system variables path. On my new Windows 10 laptop, this is found under Settings > System > About > System Info > Advanced System Settings > Environment variables button > System variables pane Edit button

Alternately, if you are lazy like I am, just create a new shortcut on your Windows desktop to sysdm.cpl and give it a name you like.

Create a Go working folder and set up a GOPATH Windows system variable

Create a new directory at the base of your C: drive called C:\gocode

Create a new Windows system variable called GOPATH with the value C:\gocode

Test Go in Cygwin

Open a new Cygwin prompt. Type the command:

go env

This should display something like the following:

In Cygwin, clone Hyperledger Fabric and the Fabric Certificate Authority with Git, clone the Java SDK, tweak the Vagrant config file and build the Linux Ubuntu virtual image

Open a Cygwin prompt. Here we go ! The commands are as follows:

cd /cygdrive/c/gocode

mkdir fabric_java_latest

cd /cygdrive/c/gocode/fabric_java_latest

git config –global core.autocrlf false

git clone  https://github.com/hyperledger/fabric.git

git clone  https://github.com/hyperledger/fabric-ca.git

# here we specify the latest and greatest commit level. The commit level hashes are taken from the bottom of web page https://github.com/hyperledger/fabric-sdk-java

# if you substitute commit levels, YOUR MILEAGE MAY VARY.

cd  fabric-ca

git reset –hard 73a42156f637702b2ac8d4ceae0d663c4c688ed4

cd ../fabric

git reset –hard e182390035ec207535c7ba0f5df70dc702057e18

cd devenv

# Get the latest Java SDK in this folder, which will be synced between Windows and the virtual image:

git clone https://github.com/hyperledger/fabric-sdk-java.git

# In this folder you need to edit the Vagrantfile configuration file.

# Add the following block:

config.vm.network :forwarded_port, guest: 7056, host: 7056, id: “java7056”, host_ip: “localhost”, autocorrect: true

config.vm.network :forwarded_port, guest: 7058, host: 7058, id: “java7058”, host_ip: “localhost”, autocorrect: true

config.vm.network :forwarded_port, guest: 8051, host: 8051, id: “java8051”, host_ip: “localhost”, autocorrect: true

config.vm.network :forwarded_port, guest: 8053, host: 8053, id: “java8053”, host_ip: “localhost”, autocorrect: true

config.vm.network :forwarded_port, guest: 8054, host: 8054, id: “java8054”, host_ip: “localhost”, autocorrect: true

config.vm.network :forwarded_port, guest: 8056, host: 8056, id: “java8056”, host_ip: “localhost”, autocorrect: true

config.vm.network :forwarded_port, guest: 8058, host: 8058, id: “java8058”, host_ip: “localhost”, autocorrect: true

# Replace the config.vm.synced_folder  block with the following:

config.vm.synced_folder “..”, “#{SRCMOUNT}”

config.vm.synced_folder “..”, “/opt/gopath/src/github.com/hyperledger/fabric”

config.vm.synced_folder “fabric-sdk-java/src/test/fixture/sdkintegration”, “/opt/gopath/src/github.com/hyperledger/fabric/sdkintegration”

config.vm.synced_folder ENV.fetch(‘LOCALDEVDIR’, “..”), “#{LOCALDEV}”

Optionally change the name of the Virtualbox image so it isn’t a duplicate:

vb.name = “hyperledger-java-latest”

Save the Vagrantfile file. Here is my copy of this file, note that I had to add a .doc file extension so WordPress would accept it. You should view it in Notepad++:

Vagrantfile

The rubber meets the road:

vagrant up

The following command will confirm it ran fine by showing 0:

echo $?

Here is the complete listing in a text file. I had to give it a “doc” file extension so WordPress would accept to insert it here, you should view it in Notepad++:

listing_vagrant_up_text_file

Next we connect to our running image using SSH with the command:

vagrant ssh

Check the folders shared between Windows and the Linux virtual image

Create 3 text files in Windows at the following locations containing « test test test » :

C:\gocode\fabric_java_latest\fabric\a.txt

C:\gocode\fabric_java_latest\fabric-ca\b.txt

C:\ gocode\fabric_java_latest\fabric\devenv\fabric-sdk-java\src\test\fixture\sdkintegration\c.txt

Next, in Cygwin , do the following commands:

pwd

cat a.txt

cat ../fabric-ca/b.txt

cat ./sdkintegration/c.txt

# You should see “test test test” as pictured below.

Build Fabric and Fabric-ca inside Docker microservices running on the Linux virtual image

Open a new Cygwin console. Type:

cd /cygdrive/c/gocode/fabric_java_latest/fabric/devenv

vagrant up

vagrant ssh

# In the Linux virtual image, do:

make docker
echo $?
cd ../fabric-ca
make docker
echo $?
cd ../fabric/sdkintegration

# this command will launch the Docker containers:
docker-compose down;  rm -rf /var/hyperledger/*; docker-compose up --force-recreate

At this point, you are up and running !

It says below:

Creating network “sdkintegration_default” with the default driver
Creating ca_peerOrg1
Creating ca_peerOrg2
Creating orderer.example.com
Creating peer0.org2.example.com
Creating peer0.org1.example.com
Creating peer1.org2.example.com
Creating peer1.org1.example.com
Attaching to ca_peerOrg1, ca_peerOrg2, orderer.example.com, peer0.org2.example.com, peer0.org1.example.com, peer1.org2.example.com, peer1.org1.example.com

Building the fabric-sdk-java and running the unit tests

We leave the docker containers (above) running, or we restart them if they have been stopped cleanly with CTRL-C. Now we are going to open a new Cygwin prompt AS AN ADMINISTRATOR, side-by-side with the first one.

In it we will do commands:

cd /cygdrive/c/gocode/fabric_java_latest/fabric/devenv/

vagrant ssh

# Now back in the virtual image, we do commands:

cd /opt/gopath/src/github.com/hyperledger/fabric/devenv/fabric-sdk-java
mvn install

This will build the SDK and run a few tests:

Tests will be run after the build. There will be a few java.io.NotSerializableExceptions as well as “Could not verify signature” verbose warnings, but ultimately, the tests will all run without error. Ultimately, we will obtain:

[INFO] Installing /opt/gopath/src/github.com/hyperledger/fabric/devenv/fabric-sdk-java/target/fabric-sdk-java-1.0.0-alpha2.jar to /home/ubuntu/.m2/repository/org/hyperledger/fabric-sdk-java/fabric-sdk-java/1.0.0-alpha2/fabric-sdk-java-1.0.0-alpha2.jar
[INFO] Installing /opt/gopath/src/github.com/hyperledger/fabric/devenv/fabric-sdk-java/pom.xml to /home/ubuntu/.m2/repository/org/hyperledger/fabric-sdk-java/fabric-sdk-java/1.0.0-alpha2/fabric-sdk-java-1.0.0-alpha2.pom
[INFO] ————————————————————————
[INFO] BUILD SUCCESS
[INFO] ————————————————————————
[INFO] Total time: 01:19 min
[INFO] Finished at: 2017-05-16T19:36:18+00:00
[INFO] Final Memory: 35M/383M
[INFO] ————————————————————————

Here are the complete listings for the “mvn install” and “docker compose” commands. Once again I saved these text files with a .doc file extension so WordPress would like it. You should look at them in Notepad++:

mvn install

docker compose

Running the end-to-end Java integration tests

In the same Cygwin prompt as the previous section, do the command:

mvn failsafe:integration-test -DskipITs=false

This will give us:

[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent! The file encoding for reports output files should be provided by the POM property ${project.reporting.outputEncoding}.
[INFO] ————————————————————————
[INFO] BUILD SUCCESS
[INFO] ————————————————————————
[INFO] Total time: 37.512 s
[INFO] Finished at: 2017-05-16T21:02:19+00:00
[INFO] Final Memory: 13M/170M
[INFO] ————————————————————————

Here are the text file listings for the “mvn integration” command as well as for the containers:

mvn integration test

listing_integration_test

We’re done for now. I recommend you CTRL-C, CTRL-D, vagrant halt, exit, log out of your Windows session, log back in and 7-Zip the C:\gocode folder again.

Building the Fabric HTML documentation with Sphinx

There is an issue with Sphinx’s imgmath python extension on Windows, related to versions of Sphinx and at what point in the program launch are the configured extensions validated. Here is how I worked around it.

Open a new Cygwin terminal. Edit the conf.py file.

vi /cygdrive/c/gocode/fabric_java_latest/fabric/docs/source/conf.py

Add a # comment character at the beginning of line:

‘sphinx.ext.imgmath’,

as below:

# Save in vi with:

:wq

cd /cygdrive/c/gocode/fabric_java_latest/fabric/docsecho

make html

As indicated, the web pages are created under

C:\gocode\fabric_java_latest\fabric\docs\build\html\

OK, all done. Now you are ready to get started with Java on the Hyperledger Linux virtual image.

(end)

Bertrand Szoghy, 2017-05, Quebec City.