Building the Hyperledger Fabric VM and Docker Images version 1.0.5 from scratch, then running the fabric-java-sdk version end to end test against it

Hello,

Introduction

I will  repeat the official procedure (which I find a bit confusing) to build the Hyperledger Fabric version 1.0.5 dev VM, log in to it and then build the Docker images from scratch. Then I will download, build and run the fabric-java-sdk end to end test.

Ultimately, we will have with an up-to-date headless Linux Ubuntu virtual image. If you don’t like Ubuntu, well, you’re a bit stuck with it because the VM post-install bash script at (Fabric source repository) fabric/devenv/setup.sh is geared toward that distro. Of course, to actually use the Docker images in a Hyperledger Fabric private network on another distro you will only need to install a few dependencies such as Docker, Golang, Git and NodeJs and follow the steps described in the fabric-samples first-network example.

If you don’t want headless Ubuntu because you want to develop Java client apps on Eclipse (or IBM Rational Application Developer), check out my procedure here to add a graphical UI to the VM along with Eclipse.

Please note this procedure was updated on 2017-11-22 because some 1.0.2 packages only two months old were deprecated and archived by the Hyperledger team and can no longer be downloaded, preventing the Docker Zookeeper image from being built. I personally do not agree with this archiving approach (!)

 

Before you can get started

You will first need to install Vagrant, Oracle Virtualbox, Golang and Cygwin. For Windows, follow the procedure here from section «Enable virtualization in the BIOS » all the way to section « Test Go in Cygwin » inclusively. You will need a quiet few hours to do so.

This procedure requires a computer with sufficient RAM memory (check the requirement in the Vagrantfile, currently more than 4GB is required).

 

The procedure

As described on page https://github.com/hyperledger/fabric-sdk-java, in a Cygwin terminal we build the Ubuntu virtual image from Hyperledger Fabric 1.0.5 sources, then build the fabric Docker images:

# the following procedure last tested end to end on 2017-11-22
cd /cygdrive/c/gocode
mkdir fabric_1.0.5
cd fabric_1.0.5

# get the sources to build VM in which we will build the Docker images
git clone  https://github.com/hyperledger/fabric.git
git clone  https://github.com/hyperledger/fabric-ca.git

# update the sources to latest version 1.0.5 tags
# if you use TortoiseGit, you can find this tag by right-clicking 
# in Windows Explorer on the git-pulled "fabric" folder, then
# doing in the popup menu TortoiseGit > Switch/Checkout,
# then in the dialog that comes up, select the "Commit" radio
# button > click the matching "..." button,
# then in the next dialog window highlight an entry marked 
# "release" or a version you want, the tag you need is in the
# first line of the tab underneath after "SHA-1: "
cd  fabric-ca
git reset --hard 26110c00ffe5409f27e6de2079cd98e9d1be7a3d
cd ../fabric
git reset --hard b19580a4a72aecc0f7ac54519f0c9e5092f0d026

# change to sub-folder where Vagrantfile is
cd devenv

# launch VM, it will update itself and install dependencies as per 
# setup.sh referred to in Vagrantfile
vagrant up
# the previous command can easily take half an hour to complete 
# on a regular connection, patience is required here.

# in case something kernel-related was updated:
vagrant reload 

# Had an issue here where Vagrant had trouble starting the VM on
# Windows 7, even with 16GB of RAM. A Windows reboot solved the problem. 

# For some reason Golang was not yet on the path in the VM at 
# this point. To resolve, I did:
vagrant provision
vagrant reload

# log in to the Ubuntu virtual machine:
vagrant ssh

# provisioning still did not do its job right, had to do the following:
sudo apt-get update
sudo apt-get dist-upgrade
exit
vagrant box update
vagrant reload
# note that for building the Kafka Docker image later, a whack load 
# more packages will be downloaded and installed... "It ain't 
# over 'til it's over." 

# Finally, now let's get down to business.
vagrant ssh
# still get a "Danger, Will Robinson" warning on login, am ignoring !!!

# check that you are in folder:
# /opt/gopath/src/github.com/hyperledger/fabric
pwd

# build fabric docker images (be very patient):
make clean
make docker

# make sure the build was successful, following should display "0":
echo $?

# build fabric-ca docker images
cd ../fabric-ca
make clean
make docker

# make sure build was successful, following should display "0":
echo $?

# 2017-11-25 all Docker images were built with no error.

exit
vagrant halt

# In Notepad++, for fabric-java-sdk testing, add in 
# Vagrantfile right after line 43:
config.vm.network :forwarded_port, guest: 7056, host: 7056
config.vm.network :forwarded_port, guest: 7058, host: 7058
config.vm.network :forwarded_port, guest: 8051, host: 8051
config.vm.network :forwarded_port, guest: 8053, host: 8053
config.vm.network :forwarded_port, guest: 8054, host: 8054
config.vm.network :forwarded_port, guest: 8056, host: 8056
config.vm.network :forwarded_port, guest: 8058, host: 8058
config.vm.network :forwarded_port, guest: 7059, host: 7059

#  In Notepad++, add in Vagrantfile between the two lines (don't 
# comment out the lines like I do below):
#####config.vm.synced_folder "..", "/opt/gopath/src/github.com/hyperledger/fabric"
config.vm.synced_folder "fabric-sdk-java/src/test/fixture/sdkintegration", "/opt/gopath/src/github.com/hyperledger/fabric/sdkintegration"
#####config.vm.synced_folder ENV.fetch('LOCALDEVDIR', ".."), "#{LOCALDEV}"

# save Vagrantfile and close Notepad++

# Now back in the Cywin prompt:
# IMPORTANT, must be under /cygdrive/c/gocode/fabric_1.0.2/fabric/devenv : 
cd /cygdrive/c/gocode/fabric_1.0.5/fabric/devenv
git clone https://github.com/hyperledger/fabric-sdk-java.git
cd fabric-sdk-java

# Use the TortoiseGit trick I describe above to get the latest
# commit tag... 
# Reset to latest version "master"
git reset --hard a8d89513f554812eea70cc4362c40af4fd1e0d60

# back to devenv folder:
cd ..
vagrant up
vagrant ssh
# still get a "Danger, Will Robinson" warning on login, am ignoring !!!

# add variable JAVA_HOME
sudo vi /etc/profile
# ajouter:
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export PATH=$JAVA_HOME/bin:$PATH

# source /etc/profile
. /etc/profile

# test we have our variable, Java and Maven:
echo $JAVA_HOME
java -version
mvn -version

cd sdkintegration
# launches fabric:
docker-compose down;  rm -rf /var/hyperledger/*; docker-compose up --force-recreate 

# 2017-11-25 network successfully up!

 

Next I built some of the fabric-java-sdk artefacts with the following steps, all successful. Then I run the fabric-java-sdk end to end tests:

# open a new Cygwin terminal:
cd /cygdrive/c/gocode/fabric_1.0.5/fabric/devenv
vagrant ssh
# still get a "Danger, Will Robinson" warning on login, am ignoring !!!

# build the jar /vagrant/fabric-sdk-java/target/fabric-sdk-java-1.0.1.jar
cd /vagrant/fabric-sdk-java
mvn install

#### at this point in the other terminal with fabric running, 
#### the following lines appear:
## peer0.org1.example.com    | 2017-09-21 13:56:57.747 UTC [endorser] ProcessProposal -> DEBU 1a6 Entry
## peer0.org1.example.com    | 2017-09-21 13:56:57.747 UTC [protoutils] ValidateProposalMessage -> DEBU 1a7 ValidateProposalMessage starts for signed proposal 0xc4202bb680
## peer0.org1.example.com    | 2017-09-21 13:56:57.747 UTC [endorser] ProcessProposal -> DEBU 1a8 Exit

# Exceptions will appear, but they are expected and part of the tests, 
# the important thing is to see:
# Results :
# Tests run: 307, Failures: 0, Errors: 0, Skipped: 3
# and:
# [INFO] BUILD SUCCESS
# build /opt/gopath/src/github.com/hyperledger/fabric/core/chaincode/shim/java/build/libs/shim-client-1.0.jar
cd /opt/gopath/src/github.com/hyperledger/fabric/core/chaincode/shim/java
gradle build

cd /opt/gopath/src/github.com/hyperledger/fabric/sdkintegration/javacc/example_cc
# ref https://stackoverflow.com/questions/39519586/shim-client-1-0-jar-missing-when-compiling-the-java-chain-code
gradle -b build.gradle build

vi ./pom.xml
# add missing dependency in pom.xml in the same folder (yes, both gradle
# and maven operations in this same folder):
<dependency>
 <groupId>javax.json</groupId>
 <artifactId>javax.json-api</artifactId>
 <version>1.1.0-M1</version>
</dependency>

mvn dependency:resolve
mvn install

# here I get:
# Results :
# Tests run: 307, Failures: 0, Errors: 0, Skipped: 3
# run the fabric-java-sdk end to end tests:
cd /vagrant/fabric-sdk-java
mvn failsafe:integration-test -DskipITs=false

 

Final result 2017-11-22:

Sans titre

Which actually is fine.

Final note: This was harder to do with version 1.0.5 than 1.0.2 two months ago.

 

 

 

 

Advertisements

grpc-java programs using an Elliptic Curve certificate for SSL communication

Introduction

RSA cryptography based on the mathematical problem of factoring the product of two large prime numbers. Elliptic Curve Cryptography (ECC) is based on the algebraic structure of elliptic curves over finite fields. ECC requires smaller keys compared to RSA to provide equivalent security. So an ECC 256bit key is stronger than a 256bit RSA key.

In my last post, “Java gRPC client and server using secure HTTP/2 channels on the Hyperledger Fabric virtual machine“, I could have just as well used an Elliptic Curve certificate and private key instead of RSA to allow my two gRPC Java programs to communicate with each other securely over SSL/TLS.

In this post, I will demonstrate how to generate the ECC certificate and key, modify the Java examples and run them again.

 

Procedure

First, start our Hyperledger Fabric virtual image and log into it by SSH. Open a Cygwin prompt in Windows, do :

 cd /cygdrive/c/gocode/fabric_java_latest/fabric/devenv

vagrant up

vagrant ssh

Once in Linux, do:

cd /devenv/nodecode/certs

It’s important that the computer the TstServiceClient.java program will try to connect to, i.e. “hyperledger-devenv“, be found in the Elliptic Curve certificate we will generate.

15.png

It is also important that this computer name be resolved. A good way of ensuring the latter is to add a line in the file /etc/hosts:

 127.0.0.1  hyperledger-devenv

Next, do the following single OpenSSL command to generate the Elliptical Curve certificate and private key:

openssl req -new -x509 -nodes -newkey ec:<(openssl ecparam -name secp256r1) -keyout ec.key -out ec.crt -days 3650

You will be prompted to enter information. It is critical that you enter the correct value for “Common Name” (indicated in red below):

using curve name prime256v1 instead of secp256r1
Generating a 256 bit EC private key
writing new private key to ‘ec.key’
—–
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter ‘.’, the field will be left blank.
—–
Country Name (2 letter code) [AU]:CA
State or Province Name (full name) [Some-State]:Quebec
Locality Name (eg, city) []:Quebec City
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Bertrand Szoghy
Organizational Unit Name (eg, section) []:dev
Common Name (e.g. server FQDN or YOUR name) []:hyperledger-devenv
Email Address []:bertrandszoghy@gmail.com

10.png

Now, log in to the virtual machine through the Virtualbox window as user ubuntu:

10.png

Next, do the command to launch the graphical user interface:

sudo startxfce4&

Double-click to start Eclipse Neon, accept the workspace.

2.png

Expand our “JavaGrpc” Maven project.

Open file com.wordpress.bertranszoghy.grpc.TstServiceServer.java and change lines 11 and 12 from:

File cert = new File(“/opt/gopath/src/github.com/hyperledger/fabric/devenv/nodecode/certs/server.crt”);
File key = new File(“/opt/gopath/src/github.com/hyperledger/fabric/devenv/nodecode/certs/key.pem”);

4.png

to:

File cert = new File(“/opt/gopath/src/github.com/hyperledger/fabric/devenv/nodecode/certs/ec.crt”);
File key = new File(“/opt/gopath/src/github.com/hyperledger/fabric/devenv/nodecode/certs/ec.key”);

11.png

Next, open file com.wordpress.bertranszoghy.grpc.TstServiceClient.java and change line 20 from:

.trustManager(new File(“/opt/gopath/src/github.com/hyperledger/fabric/devenv/nodecode/certs/server.crt”)).build())

6.png

to:

.trustManager(new File(“/opt/gopath/src/github.com/hyperledger/fabric/devenv/nodecode/certs/ec.crt”)).build())

12.png

Next, we clean the project by clicking on the project name “JavaGrpc” > menu Project > Clean…

8.png

Next, right-click on the “JavaGrpc” project name > Build Project

6.png

 

Start the Java gRPC server

In Eclipse, right-click on file TstServiceServer.java > Run As > Java Application

7

Server start on port 7777 is displayed in the console, you can ignore the warning:

8.png

Leave the server running.

 

Run the Java gRPC client

Next, in Eclipse, right-click on file TstServiceClient.java > Run As > Java Application

9.png

As before, the communication over SSL/TLS is successful:

14.png

(end of post)

Bertrand Szoghy, 2017-06, Quebec City.

 

Java gRPC client and server using secure HTTP/2 channels on the Hyperledger Fabric virtual machine

Introduction

To recap,

In my first post, I built a Hyperledger Fabric headless Ubuntu Linux virtual image from official project definition files hosted on Github on my Windows host using Vagrant and Cygwin. I successfully ran the unit and integration tests of the latest version 1.0 (alpha) of the fabric-sdk-java.

In my second post, I installed an XServer on that Hyperledger Fabric virtual image, Eclipse Neon, and was ableto launch the fabric-sdk-java end-to-end integration tests with break points, and step through the code. I also threw in the installation of the IBM WebShere Application Server version 9.

In my third post, I demonstrated how to back up the Hyperledger Fabric virtual image on my desktop and restored it to my laptop along with all the folders synchronized between Windows host and Linux vm.

In my fourth post, I linked to the new much needed “IBM Blockchain For Dummies” PDF.

In my fifth post, I linked to the video and companion PDF of Christian Cachin’s Consensus 2017 presentation “Blockchain, cryptography and concensus” which explains Hyperledger Fabric in clear technical terms.

In my sixth post, I wrote a tutorial demonstrating a gRPC client communicating with a gRPC server through SSL (TLS) on HTTP/2. These programs were coded using protocol buffers in NodeJs, which is server-side JavaScript. These technologies are all used in Hyperledger Fabric.

In this seventh post, I will demonstrate once more a gRPC client communicating with a gRPC server through SSL (TLS) on HTTP/2, but this time it will all be coded in Java using that very same Eclipse Neon IDE I set up in the second post on the Hyperledger virtual machine. To make it really interesting, we will re-use the same protocol buffer message defined for NodeJs as well as the SSL certificates we created earlier. Finally, I will demonstrate my NodeJs gRPC client talking to my Java gRPC server as well as my Java gRPC client talking to my NodeJs gRPC server securely.

Assumptions

  • You have read my previous blog posts in order;
  • You are not daunted by Java EE, Linux command lines, or Eclipse.

The steps I will describe here could probably be reproduced  on any Eclipse Neon Java EE environment.

 

Concerning the SSL Certifates Generated Using OpenSSL

Do not do the same mistake I did. I have a version of OpenSSL installed on my Windows host (version 0.9.8zf 19 Mar 2015) and another installed on the Hyperledger Fabric Linux VM.

I should warn against “back-and-forthing” between host and vm when generating certificates with OpenSSL. It’s important to run all the OpenSSL commands using the same executable, either on Windows or on the Linux virtual machine. I ran into an issue and wasted an hour or two trying to generate the private key format understood by Java after the fact on my Linux VM. I resolved by going back to the OpenSSL version I had installed on Windows and used in the generation of the NodeJs certificates. Running the same command to generate my key.pem, the Java program suddenly  accepted the private key and I was off and running.

 

Steps We Will Follow

  • Launch the Eclipse IDE on the Hyperledger Fabric virtual image
  • Re-test the NodeJs gRPC client and gRPC server programs
  • Create the Java project in Eclipse Neon
  • Generate Java source files from the message.proto file
  • Modify the generated Java source files
  • Creating the gRPC client, server and service Java source files
  • Cleaning and building the Project
  • Running the Java gRPC server
  • Running the Java gRPC client
  • Running the NodeJs gRPC client with the Java gRPC server
  • Running the Java gRPC client with the NodeJs gRPC server

 

Launch the Eclipse IDE on the Hyperledger Fabric virtual image

Open a Cygwin prompt AS AN ADMINISTRATOR.

1.png

Launch the Hyperledger Fabric virtual image using the commands:

cd /cygdrive/c/gocode/fabric_java_latest/fabric/devenv

vagrant up

This will open the Virtualbox window. I log in the virtaul machine prompt as user ubuntu, add my password.

I am logged in to Linux in a command prompt. I do the command:

sudo startxfce4&

And my graphical user interface appears:

Sans titre.png

I double-click on the “Eclipse Neon” shortcut on the left side of the pictured desktop. Eclipse Neon starts up.

2.png

I accept the default workspace:

3.png

And I am ready to code Java:

4.png

 

Re-test the NodeJs gRPC client and gRPC server programs

In my previous sixth post, I created a separate headless Linux Ubuntu virtual machine and installed NodeJs, the various gRPC dependencies, as well as created SSL certificates. I did this on a separate image because the Hyperledger Fabric vm already comes with a lot of tools already installed. I wanted to demonstrate bootstrapping.

For this post, however, I copied the NodeJs program and certs to my Hyperledger Fabric image.  Let’s test they still work as before.

Back in Cygwin, do the following command to log in to the Hyperledger Fabric image:

vagrant ssh

I could have used a terminal in the graphical interface of the VM, but I admit the weird copy and paste in it drives me nuts. CTRL-SHIFT-V to paste simply does not register in my brain and I’ve forgotten five times how to copy. I am a very patient fellow, but not for long.

So, in Cygwin, I move to the nodecode folder by doing command:

cd devenv/nodecode

On the Linux VM, this shared folder is located at:

/opt/gopath/src/github.com/hyperledger/fabric/devenv/nodecode

5.png

On Windows, through the magic provided by Vagrant, I can access the very same folder at:

C:\gocode\fabric_java_latest\fabric\devenv\nodecode

9.png

Note the protocol buffers message .proto file, which we will re-use as-is shortly in  that the SSL certificates and keys we generated last time are all in the certs subfolder:

10.png

The file key.pem above contains the same private key as server.key, but in a PKCS8 format understood by Java. See my previous post for the OpenSSL commands used to generate these files.

I start the NodeJs gRPC server with the following command (leaving out the .js file extension):

node grpcserver

7

I leave the server running.

Back in Windows, I open up a second Cygwin prompt AS AN ADMINISTRATOR:

1.png

I connect a second time to the Hyperledger Fabric virtual image via SSH using the commands:

cd /cygdrive/c/gocode/fabric_java_latest/fabric/devenv

vagrant ssh

In Linux, I change directory once more to nodecode:

cd devenv/nodecode

And I launch the NodeJs gRPC client with:

node grpcclient

This displays the protocol buffers message sent back by the gRPC server running the service:

8.png

Meanwhile the server indicated in its own prompt that it received the id parameter provided by the client:

11.png

OK, all good. Lets go back to Eclipse Neon.

 

Create the Java project in Eclipse Neon

There are different kinds of Eclipse projects you could attempt here. In reality, there is little choice but to use Mavan. There are two reasons for this.

One, because of the complicated dependencies of a Java project using grpc. In fact, the grpc-java implementation is a standalone Github project that is a bit bleeding edge when it comes to SSL security and the new HTTP/2 binary protocol’s ALPN (application layer protocol negotiation). The short and long of it is that at the time of this writing, the netty-tcnative version 2.0.1.Final approach does not appear to be ready yet. I used the security approach recommended by grpc-java, i.e. using netty-tcnative-boringssl-static version 1.1.33.Fork16.

Two, because Maven will generate Java source files from the message.proto file pretty much magically as opposed to the command-line alternative, which is noticeably more error prone.

For both these reasons, we will create a new Maven project in Eclipse called “JavaGrpc“.

Do menu File > New > Project…

Select Maven Project and click the Next > button:

13.png

Change nothing in the next window and click the Next > button:

14.png

Change nothing in the next window and click the Next > button:

15.png

In the next window we add our project name JavaGrpc to the Artifact Id field and com.wordpress.bertrandszoghy.grpc to the Package name field and click the Finish button:

16.png

And that creates our project. Expand the src folder:

17.png

We’re going to replace the pom.xml file with the one here (click to download and open in a text editor such as Notepad++):

pom

The pom.xml contains the following dependencies:

18.png

It also contains the following plugins. Note that the fix proposed by Volkan Yazici in my second post needs to be applied here as well, i.e. I need to replace the mentions of ${os.detected.classifier} in the pom.xml file with linux-x86_64:

19.png

Next we need to ask Maven to pull all these dependencies from the Internet. This is done by right-clicking on the project name “JavaGrpc” in Eclipse > Maven > Update Project… > click the OK button in the dialog that pops up:

20.png

That should take a while to finish. The Maven dependencies will be viewable in the tree-view afterwards. There are a few and they need to be just so:

5.png

OK, our Java gRPC project is set up.

Generate Java source files from the message.proto file

Once again, I remind you this simple protocol buffers message descriptor file was created as is in my previous post for NodeJs. Not change anything in it will guarantee my Java gRPC client and server programs will easily interoperate with my NodeJs ones.

The message.proto file is simple. You can download from the link below but you will need to rename it from message.proto.doc to message.proto:

message.proto

Here is what it contains:

1.png

A reminder protocol buffer syntax is strict. If you decide to re-type the above, don’t forget the curly braces need to be just so.

Maven has a (non-negotiable) convention where .proto files should be placed, i.e. under /src/main/proto

In Eclipse, right click-on main and do New > Folder and name this folder “proto”:

2.png

Make sure the new folder is located just so:

3.png

Next, create or copy-paste the file message.proto into the folder proto:

4.png

We will do two changes here in the message.proto file. We will change the first line to read:

syntax = “proto2”

and we will add a new line to define the package we will use the source files in:

14.png

Next, right-click on the “JavaGrpc” project name > Build Project

6.png

The Maven Console shows us we have a new generated-sources folder:

8.png

Two .java source files have been generated. They are:

  • TstServiceGrpc.java
  • Message.java

15

Copy and paste these two .java files into folder:

/src/main/java/com/wordpress/bertrandszoghy/grpc

and delete them from under the generated-sources folder.

You can also go ahead and delete the file:

/src/main/java/com/wordpress/bertrandszoghy/grpc/App.java

which is just a “hello world” generated by Maven.

Next, rename your message.proto to message.proto.txt by right-clicking > Rename…

11.png

In the end your folders will look like this:

16.png

Modify the generated Java source files

We will do three changes in Message.java.

First, we will change line 125 from:

private TstCoordinates() {

17.png

to:

public TstCoordinates() {

18.png

Next, we will change line 1546 from:

private TstId() {

19

to:

public TstId() {

20.png

Finally we will change line 2010 from:

private Empty() {

21.png

to:

public Empty() {

22.png

Next, in TstServiceGrpc.java, we notice there are 4 syntax issues:

1.png

These will be fixed by commenting out the annotation @java.lang.Override in four places. You will see in the screen captures the red underlined syntax errors in the “before” screenshots disappearing in the “after” screenshots…

1- We change line 90 from:

@java.lang.Override public final io.grpc.ServerServiceDefinition bindService() {

2.png

to two lines, the first commented out:

//@java.lang.Override

public final io.grpc.ServerServiceDefinition bindService() {

3.png

2 – We change line 230 from:

@java.lang.Override

4.png

to:

//@java.lang.Override

5.png

3- We change line 247 from:

@java.lang.Override

6.png

to:

//@java.lang.Override

7.png

4- We change line 260 from:

@java.lang.Override

8.png

to:

//@java.lang.Override

9.png

I will do one final change to the file, modifying line 25 from:

private TstServiceGrpc() {}

10.png

to

protected TstServiceGrpc() {}

11.png

Don’t forget to save these changes.

 

Creating the gRPC client, server and service Java source files

We will create three new files under the folder /src/main/java/com/wordpress/bertrandszoghy/grpc :

  • TstService.java
  • TstServiceServer.java
  • TstServiceClient

All three will be POJO java files. Here they are for you to download (you will need to rename the file extension from .java.doc to .java):

TstService.java

TstServiceClient.java

TstServiceServer.java

None of these files is very long.

Here is the listing for TstService.java:

1

Here is the listing for TstServiceServer.java:

2

And the longest listing is for TstServiceClient.java because of all the System.out.printlns:

3

So there you go.

 

Cleaning and building the Project

Click on the “JavaGrpc” project name, then top menu Project > Clean… > check the “Clean projects selected below” radio button >  check the “JavaGrpc” checkbox > click the OK button.

4

Next, right-click on the “JavaGrpc” project name > Build Project

5.png

Expected result is no error in the Problems tab, as illustrated below:

6.png

As we old-timers say: “We compile! We ship! Then we test.”

 

Running the Java gRPC server

First, make sure that NodeJs server is not still running on port 7777 in that Cygwin prompt. If so, do CTRL-C to stop.

Next, in Eclipse, right-click on file TstServiceServer.java > Run As > Java Application

7

Result we get looks good. The server accepts the certificate and private key for SSL and binds to the service as well as port 7777. There is a somewhat scrary netty warning in red which is a known issue, has already been fixed, but has not been ported to the version I am using. You can safely ignore:

8.png

Leave the server running.

 

Running the Java gRPC client

Next, in Eclipse, right-click on file TstServiceClient.java > Run As > Java Application

9.png

The client sends a TstId { ‘id’: 6 } protocol buffer message using the SendCoordinates method over SSL and HTTP/2 to the server which receives it:

10.png

The Java gRPC server responds by returning a TstCoordinates message containing id 3, “Jimmy Jazz”.

Then the client send a second “List” method call without a parameter and the server responds by sending back id 4, “Black Jack”.

Here is what it looks like on the client end:

11.png

 

Running the NodeJs gRPC client with the Java gRPC server

I intentionally made it so the NodeJs gRPC server (“Bill Williams” and “Happy Golucky” — see screen capture above) would not return identical data to the Java gRPC server (“Jimmy Jazz” and “Black Jack”).

While our Java gRPC server is still running, let’s go back to the second Cygwin prompt and run the NodeJs gRPC client with command:

node grpcclient

We can see it obtains “Jimmy Jazz” and “Black Jack” back from the Java gRPC server:

12.png

 

Running the Java gRPC client with the NodeJs gRPC server

First, we don’t want the NodeJs server to fail on start because our Java server is hogging port 7777.  So in Eclipse we shut down our Java gRPC server by clicking on the red square button:

13

Next, we return to our first Cygwin prompt and launch our NodeJs gRPC server with command:

node grpcserver

7

We go back to Eclipse and run the Java gRPC client again by right-clicking on file TstServiceClient.java > Run As > Java Application

9.png

Obtained result:

14.png

And there you go.

I sure made it look easy, didn’t I ?

(end of post)

Bertrand Szoghy, 2017-06, Quebec City.

 

 

Setting up Eclipse Neon for Enterprise Java development on the Hyperledger Fabric virtual image

Introduction

In the last post, we built the fabric-sdk-java which can be found at https://github.com/hyperledger/fabric-sdk-java and ran the end-to-end Java integrated tests on the Hyperledger Linux virtual image. Now we want to be able to debug by stepping through the Java code line by line in Eclipse.

In addition, although this step is optional, we will install a Java application server because in the near future we will want to interact with Hyperledger from a Java application deployed as an Enterprise Archive.

Steps we will follow

  • Modify the Vagrant configuration file to open VirtualBox on virtual image start and download Eclipse
  • Install a lightweight XServer on our virtual image
  • Fix graphical user interface screen resolution issues in our virtual image with the Virtualbox client utilities
  • Install Eclipse
  • Install Firefox and a graphical text editor
  • Add an Eclipse add-on ( IBM WebSphere Developer Tools Beta ) which will allow us to install server components for the IBM application servers
  • Install the WebSphereApplication Server V9.0 Tools to Eclipse
  • Download and install IBM Installation Manager
  • Install the WebSphere Application Server V9.0 application server on our Linux virtual image
  • Install a WebSphere Application Server profile
  • Import the fabric-sdk-java project into Eclipse
  • Fix the Maven pom.xml
  • Build the project using Maven
  • Start Hyperledger
  • Step through the code

Assumptions

I assume you are not intimidated by a Linux command line.

You should have registered at IBM Bluemix for a free account.

Modify the Vagrant configuration file to open VirtualBox on virtual image start and download Eclipse

First, let’s edit our C:\gocode\fabric_java_latest\fabric\devenv\Vagrantfile

I cheat here and use Notepad++ in Windows instead of vi in Cygwin. You can have two NotePad ++ window instances open at the same time if you add:

-multiInst

here:

40.png

Back to editing the Vagrantfile.

We will add the following lines to view Virtualbox and to add some video memory:

vb.gui = true
vb.customize [“modifyvm”, :id, ‘–vram’, ’64’]

And we will modify the following line to add lots of RAM to Linux:

vb.customize [‘modifyvm’, :id, ‘–memory’, ‘16384’]

It will look like this:

41.png

Next I will inspire myself from the excellent blog post at https://blog.versioneye.com/2015/05/05/setting-up-a-dev-environment-with-vagrant/

It’s important to note as the blog points out that you could add the following commands to the vagrant provisioning script at C:\gocode\fabric_java_latest\fabric\devenv\setup.sh in order to be able to build the entire virtual image again from scratch in fully automated fashion. Right now, however, I’ve made backups and I’ll make my changes on the already built virtual image.

Open a Cygwin prompt. Go to the base folder of the virtual image:

cd /cygdrive/c/gocode/fabric_java_latest/fabric/devenv/

Download Eclipse Neon 64-bit in a folder that is synced between Windows and the Ubuntu virtual image:

wget http://mirror.math.princeton.edu/pub/eclipse//oomph/epp/neon/R3/eclipse-inst-linux64.tar.gz

42

Let’s restart our virtual image with the extra mem we added in the Vagrantfile:

vagrant reload

Log in to the virtual machine:

vagrant ssh

First we will change the root (sudo) password as well as user ubuntu’s password which will be used to log in to the virtual image through Virtualbox:

sudo su –

passwd

passwd ubuntu

exit

43.png

Install a lightweight XServer on our virtual image

Now let’s install the XServer, the locales, etc as described in the blog post mentioned above by doing the following commands one by one:

sudo apt-get install -y xfce4 virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11
sudo apt-get install gnome-icon-theme-full tango-icon-theme
sudo echo “allowed_users=anybody” > /etc/X11/Xwrapper.config
sudo echo “LANG=en_US.UTF-8” >> /etc/environment
sudo echo “LANGUAGE=en_US.UTF-8” >> /etc/environment
sudo echo “LC_ALL=en_US.UTF-8” >> /etc/environment
sudo echo “LC_CTYPE=en_US.UTF-8” >> /etc/environment
sudo apt-get install virtualbox-guest-additions-iso
sudo mkdir /mnt

sudo mkdir /mnt/cdrom

Exit from the virtual image:

exit

Restart the virtual image:

vagrant reload

In the 800×600 resolution VirtualBox window, login as user ubuntu:

44

We’re in! A command prompt, that is. Start the XServer with command:

sudo startxfce4&

This will bring up a nice small 800 x 600  graphical user interface:

46.png

Fix graphical user interface screen resolution issues in our virtual image with the Virtualbox client utilities

I don’t know about you, but I myself refuse to code in Eclipse with a small resolution like that. Let’s make our screen bigger.

In the graphical user interface, open a terminal:

47.png

Before going further, to copy and paste in this UTerm terminal, select the text and left click your mouse. To paste, click the middle button of your mouse (i.e. the scroller on my mouse).

Do commands:

sudo mount -o loop /usr/share/virtualbox/VBoxGuestAdditions.iso /mnt/cdrom

cd /mnt/cdrom

sudo ./VBoxLinuxAdditions.run

The screen will be fixed now after a virtual image restart. Log out of the graphical user interface.

In Cygwin:

vagrant reload

In the 800×600 resolution VirtualBox window, login as user ubuntu:

44

Start the XServer with command (or find it with the up arrow):

sudo startxfce4&

Wonder of wonders, the UI fills our monitor display:

48.png

Install Eclipse

In the graphical user interface, open a terminal:

47.png
mkdir /home/ubuntu/eclipse

mkdir /home/ubuntu/workspace

cd /opt/eclipse-installer/

# You must be in graphics mode for the installer to work:
sudo ./eclipse-inst

49
Select installation folder: /home/ubuntu/eclipse:

51.png

# accept licenses

# accept certificates

Select /home/ubuntu/workspace as an Eclipse workspace:

53.png

Right-click on the screen and select menu “Create Launcher…”

54.png

55.png

Double click the “Eclipse Neon” link on the desktop, accept to “Mark as executable”, launch again and here we go with Neon:

56

Install Firefox and a graphical text editor

For some reason my default web browser was glitchy, so in the graphical user interface, I opened a terminal:

47.png

And installed Firefox:

sudo apt-get install firefox

And while at it, I also installed a half decent graphical text editor:

sudo apt-get install kate

Once installed, I launched the Firefox web browser from the terminal by typing:

firefox

Add an Eclipse add-on ( IBM WebSphere Developer Tools Beta ) which will allow us to install server components for the IBM application servers

In the browser, I headed to:

https://developer.ibm.com/wasdev/downloads/#asset/tools-WebSphere_Developer_Tools_Beta

50.png

The file here was released: 16 May 2017.

And while Eclipse Neon is open, I clicked the “Install” button at the bottom of the web page and dragged it over the Eclipse workspace:

51

A “Confirm Selected Features” dialog pops up. I’m greedy, so I select all:

52

I click the Confirm > button. I get a warning IBM WebSphere Application Server Liberty Developer Tools Beta (April 2017) is not available and do I want to proceed anyway? I click Yes.

I accept the license agreements and click the Finish button. It takes a while to conclude. Long enough to go fetcch a coffee.

53.png

At the end, you will be prompted to restart Eclipse Neon, click Yes:

55

Install the WebSphereApplication Server V9.0 Tools to Eclipse

After Eclipse has restarted, click menu Help > Install WebSphere Software…

59.png

Click to install the “WebSphere Application Server V9.0 Tools”:

60

The button becomes “Install pending”. Click the Finish button:

61.png

In the next window, click the Next button:

62.png

Accept the license and click the Finish button. It takes a while:

63.png

At this point, I got a 500 Server error in the middle of the install. I clicked click menu Help > Install WebSphere Software… and restarted the install and it picked up where it left off and finished gracefully.

After completion, you are prompted to restart Eclipse. Go ahead.

64.png

Download and install IBM Installation Manager

Next, in Firefox open a new tab and surf to URL:

https://geekflare.com/ibm-im-installation-guide/

And click on the link to download IBM Installation Manager:

54.png

This led me to an IBM login window where I was able to use my free IBM Bluemix account. Or, alternately, you can create a new account here.

Once in, click to download the “IBM Installation Manager Install Kit for all x86_64 Linux versions“:

56.png

Select to Save File and click OK:

57

It takes a while:

58

Once downloaded, we will cut and paste the zip in a folder synced between windows and the virtual image. This will give us ample hard drive space to play around in. We open the containing folder:

65

Cut the zip and paste it into /opt/gopath/src/github.com/hyperledger/fabric/devenv/

66

This corresponds to your Windows folder:

C:\gocode\fabric_java_latest\fabric\devenv\

Still in the virtual image, open a terminal:

47.png

Do commands:

cd /opt/gopath/src/github.com/hyperledger/fabric/devenv

mkdir iim

cd iim

mv ../agent.installer.linux.gtk.x86_64_1.8.2000.20150303_1526.zip

unzip agent.installer.linux.gtk.x86_64_1.8.2000.20150303_1526

./install

This will launch the IBM Installation Manager UI:

67

In the next window, click the Next button:

68

Accept the license.

For disk space limitation reasons, change the Installation Manager Directory to:

/opt/gopath/src/github.com/hyperledger/fabric/devenv/IBM/InstallationManager/eclipse

69.png

In the summary window, click Install :

70

After it has completed, click the Restart Installation Manager button:

71

Install the WebSphere Application Server V9.0 application server on our Linux virtual image

At this point we are at a crossroad. To create an entyerprise Java application which will communicaate with Hyperledger, we could use different application servers. The free ones include Tomcat or IBM WebSphere Liberty.

I will be installing the full blown WebSphere Application Server V9.0. There is a free WAS Developer version available at:

https://www.ibm.com/developerworks/downloads/ws/wasdevelopers/index.html

Login at that page using your free IBM Bluemix account.

Next, go to the following web page for instructions:

https://developer.ibm.com/wasdev/downloads/#asset/WAS_traditional_for_Developers

This is what you need to look at:

72.png

In the Linux virtual image, in IBM Installation Manager which has restarted, click menu File > Preferences… as pictured below:

73.png

In the window that appears, click the Add Repository… button:

74

Enter the URL mentioned in the IBM web page above and click OK:

75

You are prompted to enter your Bluemix account login and password. Go ahead and check the checkbox.

Next, you are prompted to create a master password for the secure storage. Put user ubuntu’s Linux password here :

76.png

Click OK to close the Repositories window:

77

Next, in IBM Installation Manager click Install:

78.png

You will be prompted to update IBM Installation Manager. Click Yes:

79.png

When it’s done, you are prompted to restart IBM Installation Manager. Click the OK button:

80.png

Next, in IBM Installation Manager click Install again:

78.png

Enter the Secure Storage password which is user ubuntu’s password.

In the next window, we will check IBM WebSphere Application Server (ILAN) and subcomponents. You could actually install Liberty as well side by side. Personally, I have found it rquires too much of my patience to configure server resources in Liberty. I like the regular WAS’s administrative console better. Click the Next > button:

81

Wait a while. I the Install Packages window that comes up, click the Next > button:

82

Accept the license agreement and click the Next > button.

For disk space limitation reasons, change the Installation Manager Directory to a folder actually synced from Windows (notice the available space I have that location in the screen capture below, a bit more that the 2GB on the Linux virtual image):

/opt/gopath/src/github.com/hyperledger/fabric/devenv/IBM/IMShared

and click the Next button:

83.png

In the next window, we will change the Installation Directory (again, for disk space reasons) to:

/opt/gopath/src/github.com/hyperledger/fabric/devenv/IBM/WebSphere/AppServer

and click on the Next > button:

84.png

In the next window, I keep the English translation as is and click on the Next > button.

In the next window, I expand the tree view and select the Sample applications and click on the Next > button:

85.png

In the next summary window, click on the Install button:

86.png

This will take quite a while!

Once completed, click the Finish button:

87.png

Exit the IBM Installation Manager.

Install a WebSphere Application Server profile

In the top Linux user interface menu, click Applications > System > Profile Management Tool `:

90

The WebSphere Customization Tool launches. We’re going to create a WAS profile. Click the Create… button:

91

In the next window, click the Next > button:

92.png

In the next window, keep the “Typical profile creation” selection and click on the Next > button.

In the Administrative security, uncheck the “Enable administrative security” checkbox and click on the Next > button:

93.png

In the Profile Creation Summary window, note the profile path and profile name. Then click the Create button:

94

This takes a while to complete.

Accept to launch the First steps console and click the Finish button:

95.png

In the Fist steps window, click on the “Installation verification” link:

96.png

Looking good:

97.png

Close the window, Exit the First Steps window. Close the WebSphere Customization Toolbox 9.0 window. Close the Uterm window.

Adding WebSphere Application Server 9.0 to Eclipse

The only wondow still open is Eclipse. Let’s customize it a bit first.

Add line numbers by going to menu Window > Preferences > General > Editors  > Text Editors > check the “Show line numbers” checkbox

Add a Console tab by going menu Window > Show View > Console

Add Problems and Progress tabs by going menu Window > Show View > Other… 

In the Show View window that comes up, expand General in the tree view, select Problems and Progress and click the OK button:

99.png

Stop the automatic jumping to the Servers tab by going menu Window > Preferences > Server and uncheck “Show Servers view when server state changes”, then click OK:

4.png

Stop the automatic jumping to the Console tab when a log is added and remote console log limit by going menu Window > Preferences > Run/Debug > Console > uncheck the “Limit console output” checkbox >  uncheck “Show when program writes to standard out” > uncheck “Show when program writes to standard error“:

6

In the Servers tab, delete the Web Preview Server by right-clicking on it > Delete:

100.png

Click OK.

Next, click on the “create a new server” link in the Servers tab.

In the window that comes up, I select “WebSphere Application Server traditional V9.0” in the server type tree view, shorten the Server name a bit and click on the Next > button:

101.png

In the next window, we browse to our Installation folder which for disk space reasons we placed at:

/opt/gopath/src/github.com/hyperledger/fabric/devenv/IBM/WebSphere/AppServer

Leave the JRE as is. Click the Next > button:

102

In the next window, leave everything as is and click the Finish button:

104.png

We now have a “WAS 9.0 locahost” server in our Servers tab:

1.png

It started automatically.

Right click on “WAS 9.0 locahost” > Properties > WebSphere Application to display product version, useful port information and log location:

3.png

Click OK to close the Properties window.

Now to stop our server from automatically building and deploying, double-click on “WAS 9.0 locahost” to bring up the “WAS 9.0 localhost” top pane >  expand Publishing > select the “Never publish automatically” radio button > click the top left Save button:

7

To open the WebSphere Administration Console, right-click on “WAS 9.0 locahost”  > Administration > Run Administrative Console

8

The console will appear in a Firefox web browser window . It takes a while to come up. We set it up so we don’t have to type in a password and it goes right in without a log in screen (improvement over version 8, 7, 6…):

9.png

Import the fabric-sdk-java project into Eclipse

In Eclipse’s Enterprise Explorer pane (top left), right-click > Import > Import

10

In the Import window that comes up, expand General in the tree view and select Maven > Existing Maven Projects and click the Next > button:

21.png

In the Import Maven Projects window that comes up, click the Browse… button:

22

Click “+ Other Locations” in the next Linux dialog:

23

Click on “Computer” in the next window:

24.png

Navigate to /opt/gopath/src/github.com/hyperledger/fabric/devenv/fabric-sdk-java and click the OK button:

25.png

Keep the next window settings as is and click the Finish button:

26.png

Expand the new fabric-java-sdk project in the Enterprise Explorer tab:

27.png

Update dependencies by right-clicking on the project name > Maven > Update Project...

28.png

You can see the results in the Maven Console:

29

Fix the Maven pom.xml

The expected outcome at this point would be to just build the project the regular way and Maven would do just do its magic. This was not my experience. I caught two nasty build errors which read:

The error listed is:

1 required artifact is missing.

for artifact:
org.hyperledger.fabric-sdk-java:fabric-sdk-java:jar:1.0.0-alpha2

from the specified remote repositories:
central (https://repo.maven.apache.org/maven2, releases=true, snapshots=false)

at org.xolstice.maven.plugin.protobuf.AbstractProtocMojo.resolveBinaryArtifact(AbstractProtocMojo.java:962)
at org.xolstice.maven.plugin.protobuf.AbstractProtocMojo.execute(AbstractProtocMojo.java:489)
at org.xolstice.maven.plugin.protobuf.AbstractProtocCompileMojo.execute(AbstractProtocCompileMojo.java:31)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at org.eclipse.m2e.core.internal.embedder.MavenImpl.execute(MavenImpl.java:331)
at org.eclipse.m2e.core.internal.embedder.MavenImpl$11.call(MavenImpl.java:1362)
at org.eclipse.m2e.core.internal.embedder.MavenImpl$11.call(MavenImpl.java:1)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:176)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:112)
at org.eclipse.m2e.core.internal.embedder.MavenImpl.execute(MavenImpl.java:1360)
at org.eclipse.m2e.core.project.configurator.MojoExecutionBuildParticipant.build(MojoExecutionBuildParticipant.java:52)
at org.eclipse.m2e.core.internal.builder.MavenBuilderImpl.build(MavenBuilderImpl.java:137)
at org.eclipse.m2e.core.internal.builder.MavenBuilder$1.method(MavenBuilder.java:172)
at org.eclipse.m2e.core.internal.builder.MavenBuilder$1.method(MavenBuilder.java:1)
at org.eclipse.m2e.core.internal.builder.MavenBuilder$BuildMethod$1$1.call(MavenBuilder.java:115)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:176)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:112)
at org.eclipse.m2e.core.internal.builder.MavenBuilder$BuildMethod$1.call(MavenBuilder.java:105)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:176)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:151)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:99)
at org.eclipse.m2e.core.internal.builder.MavenBuilder$BuildMethod.execute(MavenBuilder.java:86)
at org.eclipse.m2e.core.internal.builder.MavenBuilder.build(MavenBuilder.java:200)
at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:735)
at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:206)
at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:246)
at org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:301)
at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:304)
at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:360)
at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:383)
at org.eclipse.core.internal.resources.Workspace.buildInternal(Workspace.java:487)
at org.eclipse.core.internal.resources.Workspace.build(Workspace.java:406)
at org.eclipse.ui.actions.BuildAction$1.runInWorkspace(BuildAction.java:285)
at org.eclipse.core.internal.resources.InternalWorkspaceJob.run(InternalWorkspaceJob.java:39)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
Caused by: org.apache.maven.artifact.resolver.MultipleArtifactsNotFoundException: Missing:
———-
1) com.google.protobuf:protoc:exe:${os.detected.classifier}:3.0.0

Try downloading the file manually from the project website.

Then, install it using the command:
mvn install:install-file -DgroupId=com.google.protobuf -DartifactId=protoc -Dversion=3.0.0 -Dclassifier=${os.detected.classifier} -Dpackaging=exe -Dfile=/path/to/file

Alternatively, if you host your own repository you can deploy the file there:
mvn deploy:deploy-file -DgroupId=com.google.protobuf -DartifactId=protoc -Dversion=3.0.0 -Dclassifier=${os.detected.classifier} -Dpackaging=exe -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]

Path to dependency:
1) org.hyperledger.fabric-sdk-java:fabric-sdk-java:jar:1.0.0-alpha2
2) com.google.protobuf:protoc:exe:${os.detected.classifier}:3.0.0

I was baffled and Google this a bit. I read Volkan Yazici’s blog post on “Compiling Protocol Buffers Sources in Maven” at http://vlkan.com/blog/post/2015/11/27/maven-protobuf/

I emailed Volkan and asked him if he had an idea and he very kindly wrote back, saying:

I guess Eclipse could not resolve the ${os.detected.classifier} property dynamically injected by the os-maven-plugin. Try replacing occurrences of ${os.detected.classifier} with their hardcoded values, e.g., linux-x86_64. (See https://github.com/trustin/os-maven-plugin for list of available options).

So I edited my pom.xml, replacing two instances of ${os.detected.classifier} with linux-x86_64. Here attached is that modified pom.xml file renamed ad “pom.doc” so WordPress would allow me to add it. It is best viewed in Notepad++:

pom

Build the project using Maven

At this point, what I should have done is “menu Project > Clean…” in Eclipse followed by a “menu Project > Build Project” and that should have built fine. But I was distracted, and skipped the clean step and got one error:

An IO error occured (org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile-custom:default:generate-sources)

org.apache.maven.plugin.MojoExecutionException: An IO error occured
(…)  Caused by: java.io.IOException: Directory /opt/gopath/src/github.com/hyperledger/fabric/devenv/fabric-sdk-java/target/protoc-dependencies/6056016fa52e6e795c6b7e22bc022718/google unable to be deleted.

No problem, I tell myself. I’ll just close Eclipse, open a terminal in Linux and type command:

sudo rm -Rf /opt/gopath/src/github.com/hyperledger/fabric/devenv/fabric-sdk-java/target/protoc-dependencies/6056016fa52e6e795c6b7e22bc022718/google

Restart Eclipse and build. That will be easy, right?

So after quite a few minutes feebly struggling to copy and paste into the virtual image terminal, I followed the advice at https://askubuntu.com/questions/617629/how-to-enable-cut-or-copy-paste-in-xterm and in the Applications > Accessories > Kate created a new text file called /home/ubuntu/XTerm containing:

XTerm*Background: black
XTerm*Foreground: green
XTerm*SaveLines: 2000
XTerm*faceName: Ubuntu mono
XTerm*faceSize: 12
XTerm*rightScrollBar: true
XTerm*ScrollBar: true
XTerm*scrollTtyOutput: false
XTerm*VT100.Translations: #override \
      Shift CtrlV: insert-selection(CLIPBOARD) \n\ 
      Shift CtrlV: insert-selection(PRIMARY) \n\ 
      Shift: select-start() \n\ 
      Shift: select-extend() \n\ 
      Shift: select-end(CLIPBOARD) \n\

I saved the text file, launched a new terminal and was able to paste half-decently with SHIFT-CTRL-V. Thank you, Simon!

2.png

Restarted Eclipse. Did a “menu Project > Clean…” in Eclipse followed by a “menu Project > Build Project” and that worked, I got a clean build.

3

Start Hyperledger Fabric

Referring to my previous blog post, I see that I have to start the pers using the following commands in a Linux terminal:

cd /opt/gopath/src/github.com/hyperledger/fabric/sdkintegration

docker-compose down;  rm -rf /var/hyperledger/*; docker-compose up --force-recreate

We leave this terminal window open.

Step through the code

Taking a look at my previous post, from the end-to-end integration test logs in file mvn-integration-test.doc, we can see where I could place some break points:

5.png

I decided Chain.java line 329 is interesting so that’s where I placed a breakpoint:

6.png

I launch the Debugger by right-clicking  on org.hyperledger.fabric.sdkintegration.End2endIT.java > Debug As… > JUnit Test

7.png

And the code execution pauses at the expected break point.

To celebrate success, I created a screen capture movie of me stepping through the Java code in Debug mode viewable on Youtube here:

https://www.youtube.com/watch?v=5bAimyuEy8I

(end of post)

Bertrand Szoghy, 2017-05, Quebec City.

Java SDK for Hyperledger Fabric 1.0 (IBM Blockchain) — Setting Up the Environment

Hello,

IBM Blockchain is based the Linux Foundation’s open source Hyperledger Fabric project, although not all implementations of Hyperledger Fabric version 1.0 ALPHA are also IBM Blockchain. At the time of this writing (May 2017), the Hyperledger Fabric project is at version 1.0-alpha, targeting a launch for version 1.0 proper sometime in mid to late June.

Update 2018-05-03 fixed a Powershell command typo caused by WordPress converting to HTML special characters.

What we intend to build

After a quick overview of what is this blockchain technology, I will step through the building of the recommended Linux Virtual machine on a Windows host, start Docker microservices running Hyperledger Fabric on that virtual image, build the fabric-sdk-java which can be found at https://github.com/hyperledger/fabric-sdk-java and run the java unit tests. Next, we will run the end to end Java integration tests. Finally, we will build the HTML documentation with Sphinx.

If you are faint-of-heart but nevertheless curious, complete text file listings of the operations will be provided below. If you want you, can simply go over those files first if building all this seems a bit too daunting.

Assumptions

If you intent to replicate, here are my assumptions:

You know how to set a new Windows system variable through the Control Panel > System. You have administrator privileges on your Windows box.

You are not daunted by command prompts.

You like to edit configuration files in vi on Linux.

You looked at the docs: https://hyperledger-fabric.readthedocs.io/en/latest/index.html

Ramp-up references and cool videos

There is a free self-paced course offered by IBM at  https://developer.ibm.com/courses/all-courses/blockchain-for-developers/ which is helpful to understand the basics of blockchain and Hyperledger. You can follow it by logging in using a (free) IBM Bluemix account. But mostly this 10 hour course refers to Youtube videos, so in short, here below is just the good stuff. Some of the videos have dodgy titles which explains why they are so badly indexed on Google.

Use cases (9 minutes)

What is a fabric? (5 minutes)

Participants in a blockchain (3 minutes)

How Apps Use the Ledger (4 minutes)

Transactions (8 minutes, 5 stars!)

Integrating with existing systems (3 minutes)

Hyperledger Fabric Architecture (9 minutes)

Bluemix demo:

New terms you might want to remember:

Overall Steps we will follow:

  • Enable virtualization in the BIOS
  • Install Chocolatey with Powershell
  • Install Vagrant and Virtualbox with Powershell
  • Install Cygwin with Powershell
  • Install GnuPg on Cygwin using Powershell
  • Install OpenSSH on Cygwin using Powershell
  • Install Rsync on Cygwin using Powershell (this step might be optional)
  • Install Ncurses on Cygwin using Powershell (this step might be optional)
  • Install Git on Cygwin using Powershell
  • Install Python on Cygwin using Powershell
  • Install make on Cygwin using Powershell
  • Install Sphinx on Cygwin using Powershell
  • Install the Go programming language on Windows
  • Create a Go working folder and set up a GOPATH Windows system variable
  • Test Go in Cygwin
  • In Cygwin, clone Hyperledger Fabric and the Fabric Certificate Authority with Git, clone the Java SDK, tweak the Vagrant config file and build the Linux Ubuntu virtual image
  • Check the folders shared between Windows and the Linux virtual image
  • Build Fabric and Fabric-ca inside Docker microservices running on the Linux virtual image
  • Building the fabric-sdk-java and running the unit tests
  • Running the end-to-end Java integration tests
  • Building the Fabric HTML documentation with Sphinx

Enable virtualization in the BIOS

Required for Virtualbox.

Boot the PC go into the BIOS. Set the Virtualization and VT-d to enabled. I am running a Windows 7 box.

bios

Install Chocolatey with Powershell

Chocolatey is a series of Powershell automation scripts which will help us install Vagrant, Virtualbox and Cygwin. Reference: chocolatey.org

Open a Powershell prompt AS AN ADMINISTRATOR. Commands:

Get-ExecutionPolicy

If the response is « Restricted » as here below :

01

Then do both these commands (the Chocolatey site seems to say either is required, but I found both are):

Set-ExecutionPolicy AllSigned

Set-ExecutionPolicy Bypass

(and accept with Y for each)

Install Chocolatey with:

iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))

(and accept with Y)

Close the Powershell prompt.

Install Vagrant et Virtualbox with Powershell

Open a new Powershell prompt AS AN ADMINISTRATOR. Do command:

cinst vagrant virtualbox

02

(and accept with Y)

Here a Windows reboot is required..

Install Cygwin with Powershell

Open a new Powershell prompt AS AN ADMINISTRATOR. Do command:

cinst cyg-get

03

(and accept with Y)

Close the Powershell prompt.

Cygwin has now been installed under C:\tools but we will need to add a couple of features to it. Admittedly, a few of these commands could have been lumped together. They were the result of much hit and miss.

Install various useful development tools in Cygwin using Powershell

Open a new Powershell prompt AS AN ADMINISTRATOR. Do command:

cyg-get gnupg openssh ncurses rsync git make python python-sphinx

Install the Go programming language on Windows

Google’s Go programming language will be required BOTH on Windows and in the Linux virtual image. We have to worry about the former, the latter is handled for us. FYI, Go is also the programming language used to program the microservice container tool Docker.

Download Go from https://golang.org/dl/

07

Install the MSI:

08.png

Accept all suggested values.

Add C:\Go\bin to the Windows system variables path. On my new Windows 10 laptop, this is found under Settings > System > About > System Info > Advanced System Settings > Environment variables button > System variables pane Edit button

Alternately, if you are lazy like I am, just create a new shortcut on your Windows desktop to sysdm.cpl and give it a name you like.

Create a Go working folder and set up a GOPATH Windows system variable

Create a new directory at the base of your C: drive called C:\gocode

Create a new Windows system variable called GOPATH with the value C:\gocode

09

Test Go in Cygwin

Open a new Cygwin prompt. Type the command:

go env

This should display something like the following:

10

In Cygwin, clone Hyperledger Fabric and the Fabric Certificate Authority with Git, clone the Java SDK, tweak the Vagrant config file and build the Linux Ubuntu virtual image

Open a Cygwin prompt. Here we go ! The commands are as follows:

cd /cygdrive/c/gocode

mkdir fabric_java_latest

cd /cygdrive/c/gocode/fabric_java_latest

git config –global core.autocrlf false

git clone  https://github.com/hyperledger/fabric.git

git clone  https://github.com/hyperledger/fabric-ca.git

# here we specify the latest and greatest commit level. The commit level hashes are taken from the bottom of web page https://github.com/hyperledger/fabric-sdk-java

# if you substitute commit levels, YOUR MILEAGE MAY VARY.

cd  fabric-ca

git reset –hard 73a42156f637702b2ac8d4ceae0d663c4c688ed4

cd ../fabric

git reset –hard e182390035ec207535c7ba0f5df70dc702057e18

11.png

cd devenv

# Get the latest Java SDK in this folder, which will be synced between Windows and the virtual image:

git clone https://github.com/hyperledger/fabric-sdk-java.git

# In this folder you need to edit the Vagrantfile configuration file.

# Add the following block:

config.vm.network :forwarded_port, guest: 7056, host: 7056, id: “java7056”, host_ip: “localhost”, autocorrect: true

config.vm.network :forwarded_port, guest: 7058, host: 7058, id: “java7058”, host_ip: “localhost”, autocorrect: true

config.vm.network :forwarded_port, guest: 8051, host: 8051, id: “java8051”, host_ip: “localhost”, autocorrect: true

config.vm.network :forwarded_port, guest: 8053, host: 8053, id: “java8053”, host_ip: “localhost”, autocorrect: true

config.vm.network :forwarded_port, guest: 8054, host: 8054, id: “java8054”, host_ip: “localhost”, autocorrect: true

config.vm.network :forwarded_port, guest: 8056, host: 8056, id: “java8056”, host_ip: “localhost”, autocorrect: true

config.vm.network :forwarded_port, guest: 8058, host: 8058, id: “java8058”, host_ip: “localhost”, autocorrect: true

# Replace the config.vm.synced_folder  block with the following:

config.vm.synced_folder “..”, “#{SRCMOUNT}”

config.vm.synced_folder “..”, “/opt/gopath/src/github.com/hyperledger/fabric”

config.vm.synced_folder “fabric-sdk-java/src/test/fixture/sdkintegration”, “/opt/gopath/src/github.com/hyperledger/fabric/sdkintegration”

config.vm.synced_folder ENV.fetch(‘LOCALDEVDIR’, “..”), “#{LOCALDEV}”

Optionally change the name of the Virtualbox image so it isn’t a duplicate:

vb.name = “hyperledger-java-latest”

Save the Vagrantfile file. Here is my copy of this file, note that I had to add a .doc file extension so WordPress would accept it. You should view it in Notepad++:

Vagrantfile

The rubber meets the road:

vagrant up

The following command will confirm it ran fine by showing 0:

echo $?

26

Here is the complete listing in a text file. I had to give it a “doc” file extension so WordPress would accept to insert it here, you should view it in Notepad++:

listing_vagrant_up_text_file

Next we connect to our running image using SSH with the command:

vagrant ssh

14

Check the folders shared between Windows and the Linux virtual image

Create 3 text files in Windows at the following locations containing « test test test » :

C:\gocode\fabric_java_latest\fabric\a.txt

C:\gocode\fabric_java_latest\fabric-ca\b.txt

C:\ gocode\fabric_java_latest\fabric\devenv\fabric-sdk-java\src\test\fixture\sdkintegration\c.txt

Next, in Cygwin , do the following commands:

pwd

cat a.txt

cat ../fabric-ca/b.txt

cat ./sdkintegration/c.txt

# You should see “test test test” as pictured below.

15.png

Build Fabric and Fabric-ca inside Docker microservices running on the Linux virtual image

Open a new Cygwin console. Type:

cd /cygdrive/c/gocode/fabric_java_latest/fabric/devenv

vagrant up

27

vagrant ssh

# In the Linux virtual image, do:

make docker
echo $?
cd ../fabric-ca
make docker
echo $?
cd ../fabric/sdkintegration

# this command will launch the Docker containers:
docker-compose down;  rm -rf /var/hyperledger/*; docker-compose up --force-recreate

At this point, you are up and running !

It says below:

Creating network “sdkintegration_default” with the default driver
Creating ca_peerOrg1
Creating ca_peerOrg2
Creating orderer.example.com
Creating peer0.org2.example.com
Creating peer0.org1.example.com
Creating peer1.org2.example.com
Creating peer1.org1.example.com
Attaching to ca_peerOrg1, ca_peerOrg2, orderer.example.com, peer0.org2.example.com, peer0.org1.example.com, peer1.org2.example.com, peer1.org1.example.com

28

Building the fabric-sdk-java and running the unit tests

We leave the docker containers (above) running, or we restart them if they have been stopped cleanly with CTRL-C. Now we are going to open a new Cygwin prompt AS AN ADMINISTRATOR, side-by-side with the first one.

In it we will do commands:

cd /cygdrive/c/gocode/fabric_java_latest/fabric/devenv/

vagrant ssh

# Now back in the virtual image, we do commands:

cd /opt/gopath/src/github.com/hyperledger/fabric/devenv/fabric-sdk-java
mvn install

This will build the SDK and run a few tests:

30

Tests will be run after the build. There will be a few java.io.NotSerializableExceptions as well as “Could not verify signature” verbose warnings, but ultimately, the tests will all run without error. Ultimately, we will obtain:

[INFO] Installing /opt/gopath/src/github.com/hyperledger/fabric/devenv/fabric-sdk-java/target/fabric-sdk-java-1.0.0-alpha2.jar to /home/ubuntu/.m2/repository/org/hyperledger/fabric-sdk-java/fabric-sdk-java/1.0.0-alpha2/fabric-sdk-java-1.0.0-alpha2.jar
[INFO] Installing /opt/gopath/src/github.com/hyperledger/fabric/devenv/fabric-sdk-java/pom.xml to /home/ubuntu/.m2/repository/org/hyperledger/fabric-sdk-java/fabric-sdk-java/1.0.0-alpha2/fabric-sdk-java-1.0.0-alpha2.pom
[INFO] ————————————————————————
[INFO] BUILD SUCCESS
[INFO] ————————————————————————
[INFO] Total time: 01:19 min
[INFO] Finished at: 2017-05-16T19:36:18+00:00
[INFO] Final Memory: 35M/383M
[INFO] ————————————————————————

Here are the complete listings for the “mvn install” and “docker compose” commands. Once again I saved these text files with a .doc file extension so WordPress would like it. You should look at them in Notepad++:

mvn install

docker compose

Running the end-to-end Java integration tests

In the same Cygwin prompt as the previous section, do the command:

mvn failsafe:integration-test -DskipITs=false

This will give us:

[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent! The file encoding for reports output files should be provided by the POM property ${project.reporting.outputEncoding}.
[INFO] ————————————————————————
[INFO] BUILD SUCCESS
[INFO] ————————————————————————
[INFO] Total time: 37.512 s
[INFO] Finished at: 2017-05-16T21:02:19+00:00
[INFO] Final Memory: 13M/170M
[INFO] ————————————————————————

Here are the text file listings for the “mvn integration” command as well as for the containers:

mvn integration test

listing_integration_test

We’re done for now. I recommend you CTRL-C, CTRL-D, vagrant halt, exit, log out of your Windows session, log back in and 7-Zip the C:\gocode folder again.

Building the Fabric HTML documentation with Sphinx

There is an issue with Sphinx’s imgmath python extension on Windows, related to versions of Sphinx and at what point in the program launch are the configured extensions validated. Here is how I worked around it.

Open a new Cygwin terminal. Edit the conf.py file.

vi /cygdrive/c/gocode/fabric_java_latest/fabric/docs/source/conf.py

Add a # comment character at the beginning of line:

‘sphinx.ext.imgmath’,

as below:

20.png

# Save in vi with:

:wq

cd /cygdrive/c/gocode/fabric_java_latest/fabric/docsecho

make html

21

As indicated, the web pages are created under

C:\gocode\fabric_java_latest\fabric\docs\build\html\

22.png

OK, all done. Now you are ready to get started with Java on the Hyperledger Linux virtual image.

(end)

Bertrand Szoghy, 2017-05, Quebec City.