Building the Hyperledger Fabric VM and Docker Images version 1.0.5 from scratch, then running the fabric-java-sdk version end to end test against it

Hello,

Introduction

I will  repeat the official procedure (which I find a bit confusing) to build the Hyperledger Fabric version 1.0.5 dev VM, log in to it and then build the Docker images from scratch. Then I will download, build and run the fabric-java-sdk end to end test.

Ultimately, we will have with an up-to-date headless Linux Ubuntu virtual image. If you don’t like Ubuntu, well, you’re a bit stuck with it because the VM post-install bash script at (Fabric source repository) fabric/devenv/setup.sh is geared toward that distro. Of course, to actually use the Docker images in a Hyperledger Fabric private network on another distro you will only need to install a few dependencies such as Docker, Golang, Git and NodeJs and follow the steps described in the fabric-samples first-network example.

If you don’t want headless Ubuntu because you want to develop Java client apps on Eclipse (or IBM Rational Application Developer), check out my procedure here to add a graphical UI to the VM along with Eclipse.

Please note this procedure was updated on 2017-11-22 because some 1.0.2 packages only two months old were deprecated and archived by the Hyperledger team and can no longer be downloaded, preventing the Docker Zookeeper image from being built. I personally do not agree with this archiving approach (!)

 

Before you can get started

You will first need to install Vagrant, Oracle Virtualbox, Golang and Cygwin. For Windows, follow the procedure here from section «Enable virtualization in the BIOS » all the way to section « Test Go in Cygwin » inclusively. You will need a quiet few hours to do so.

This procedure requires a computer with sufficient RAM memory (check the requirement in the Vagrantfile, currently more than 4GB is required).

 

The procedure

As described on page https://github.com/hyperledger/fabric-sdk-java, in a Cygwin terminal we build the Ubuntu virtual image from Hyperledger Fabric 1.0.5 sources, then build the fabric Docker images:

# the following procedure last tested end to end on 2017-11-22
cd /cygdrive/c/gocode
mkdir fabric_1.0.5
cd fabric_1.0.5

# get the sources to build VM in which we will build the Docker images
git clone  https://github.com/hyperledger/fabric.git
git clone  https://github.com/hyperledger/fabric-ca.git

# update the sources to latest version 1.0.5 tags
# if you use TortoiseGit, you can find this tag by right-clicking 
# in Windows Explorer on the git-pulled "fabric" folder, then
# doing in the popup menu TortoiseGit > Switch/Checkout,
# then in the dialog that comes up, select the "Commit" radio
# button > click the matching "..." button,
# then in the next dialog window highlight an entry marked 
# "release" or a version you want, the tag you need is in the
# first line of the tab underneath after "SHA-1: "
cd  fabric-ca
git reset --hard 26110c00ffe5409f27e6de2079cd98e9d1be7a3d
cd ../fabric
git reset --hard b19580a4a72aecc0f7ac54519f0c9e5092f0d026

# change to sub-folder where Vagrantfile is
cd devenv

# launch VM, it will update itself and install dependencies as per 
# setup.sh referred to in Vagrantfile
vagrant up
# the previous command can easily take half an hour to complete 
# on a regular connection, patience is required here.

# in case something kernel-related was updated:
vagrant reload 

# Had an issue here where Vagrant had trouble starting the VM on
# Windows 7, even with 16GB of RAM. A Windows reboot solved the problem. 

# For some reason Golang was not yet on the path in the VM at 
# this point. To resolve, I did:
vagrant provision
vagrant reload

# log in to the Ubuntu virtual machine:
vagrant ssh

# provisioning still did not do its job right, had to do the following:
sudo apt-get update
sudo apt-get dist-upgrade
exit
vagrant box update
vagrant reload
# note that for building the Kafka Docker image later, a whack load 
# more packages will be downloaded and installed... "It ain't 
# over 'til it's over." 

# Finally, now let's get down to business.
vagrant ssh
# still get a "Danger, Will Robinson" warning on login, am ignoring !!!

# check that you are in folder:
# /opt/gopath/src/github.com/hyperledger/fabric
pwd

# build fabric docker images (be very patient):
make clean
make docker

# make sure the build was successful, following should display "0":
echo $?

# build fabric-ca docker images
cd ../fabric-ca
make clean
make docker

# make sure build was successful, following should display "0":
echo $?

# 2017-11-25 all Docker images were built with no error.

exit
vagrant halt

# In Notepad++, for fabric-java-sdk testing, add in 
# Vagrantfile right after line 43:
config.vm.network :forwarded_port, guest: 7056, host: 7056
config.vm.network :forwarded_port, guest: 7058, host: 7058
config.vm.network :forwarded_port, guest: 8051, host: 8051
config.vm.network :forwarded_port, guest: 8053, host: 8053
config.vm.network :forwarded_port, guest: 8054, host: 8054
config.vm.network :forwarded_port, guest: 8056, host: 8056
config.vm.network :forwarded_port, guest: 8058, host: 8058
config.vm.network :forwarded_port, guest: 7059, host: 7059

#  In Notepad++, add in Vagrantfile between the two lines (don't 
# comment out the lines like I do below):
#####config.vm.synced_folder "..", "/opt/gopath/src/github.com/hyperledger/fabric"
config.vm.synced_folder "fabric-sdk-java/src/test/fixture/sdkintegration", "/opt/gopath/src/github.com/hyperledger/fabric/sdkintegration"
#####config.vm.synced_folder ENV.fetch('LOCALDEVDIR', ".."), "#{LOCALDEV}"

# save Vagrantfile and close Notepad++

# Now back in the Cywin prompt:
# IMPORTANT, must be under /cygdrive/c/gocode/fabric_1.0.2/fabric/devenv : 
cd /cygdrive/c/gocode/fabric_1.0.5/fabric/devenv
git clone https://github.com/hyperledger/fabric-sdk-java.git
cd fabric-sdk-java

# Use the TortoiseGit trick I describe above to get the latest
# commit tag... 
# Reset to latest version "master"
git reset --hard a8d89513f554812eea70cc4362c40af4fd1e0d60

# back to devenv folder:
cd ..
vagrant up
vagrant ssh
# still get a "Danger, Will Robinson" warning on login, am ignoring !!!

# add variable JAVA_HOME
sudo vi /etc/profile
# ajouter:
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export PATH=$JAVA_HOME/bin:$PATH

# source /etc/profile
. /etc/profile

# test we have our variable, Java and Maven:
echo $JAVA_HOME
java -version
mvn -version

cd sdkintegration
# launches fabric:
docker-compose down;  rm -rf /var/hyperledger/*; docker-compose up --force-recreate 

# 2017-11-25 network successfully up!

 

Next I built some of the fabric-java-sdk artefacts with the following steps, all successful. Then I run the fabric-java-sdk end to end tests:

# open a new Cygwin terminal:
cd /cygdrive/c/gocode/fabric_1.0.5/fabric/devenv
vagrant ssh
# still get a "Danger, Will Robinson" warning on login, am ignoring !!!

# build the jar /vagrant/fabric-sdk-java/target/fabric-sdk-java-1.0.1.jar
cd /vagrant/fabric-sdk-java
mvn install

#### at this point in the other terminal with fabric running, 
#### the following lines appear:
## peer0.org1.example.com    | 2017-09-21 13:56:57.747 UTC [endorser] ProcessProposal -> DEBU 1a6 Entry
## peer0.org1.example.com    | 2017-09-21 13:56:57.747 UTC [protoutils] ValidateProposalMessage -> DEBU 1a7 ValidateProposalMessage starts for signed proposal 0xc4202bb680
## peer0.org1.example.com    | 2017-09-21 13:56:57.747 UTC [endorser] ProcessProposal -> DEBU 1a8 Exit

# Exceptions will appear, but they are expected and part of the tests, 
# the important thing is to see:
# Results :
# Tests run: 307, Failures: 0, Errors: 0, Skipped: 3
# and:
# [INFO] BUILD SUCCESS
# build /opt/gopath/src/github.com/hyperledger/fabric/core/chaincode/shim/java/build/libs/shim-client-1.0.jar
cd /opt/gopath/src/github.com/hyperledger/fabric/core/chaincode/shim/java
gradle build

cd /opt/gopath/src/github.com/hyperledger/fabric/sdkintegration/javacc/example_cc
# ref https://stackoverflow.com/questions/39519586/shim-client-1-0-jar-missing-when-compiling-the-java-chain-code
gradle -b build.gradle build

vi ./pom.xml
# add missing dependency in pom.xml in the same folder (yes, both gradle
# and maven operations in this same folder):
<dependency>
 <groupId>javax.json</groupId>
 <artifactId>javax.json-api</artifactId>
 <version>1.1.0-M1</version>
</dependency>

mvn dependency:resolve
mvn install

# here I get:
# Results :
# Tests run: 307, Failures: 0, Errors: 0, Skipped: 3
# run the fabric-java-sdk end to end tests:
cd /vagrant/fabric-sdk-java
mvn failsafe:integration-test -DskipITs=false

 

Final result 2017-11-22:

Sans titre

Which actually is fine.

Final note: This was harder to do with version 1.0.5 than 1.0.2 two months ago.

 

 

 

 

Advertisements

Linux CentOs 7.3 Vagrant virtual image with GUI: fast track

Hello,

In my previous post I described how to upgrade a Vagrant CentOS image from version 7.2 to 7.3, changing languages, all the nitty gritty.

Well, immediate gratification always takes too long, so here is a shortcut: the esss/centos-7.3 Vagrant box at https://app.vagrantup.com/esss/boxes/centos-7.3

Open Cygwin and chamge to your gocode folder (as described in this previous post) and do the following commands:

esss

cd /cygdrive/c/gocode

mkdir essscentos

cd essscentos

vagrant init esss/centos-7.3 –box-version 1.0.0

Okay, so now open the following new file in Notepad++: C:\gocode\essscentos\Vagrantfile

Uncomment the GUI block starting at line 53 and raise the memory so it looks like this:

note.png

Now go back to Cygwin and do command:

vagrant up

Log in to the GUI is user vagrant and password vagrant, as usual.

Enjoy!