The path to get somewhere depends on where you come from, and I often find myself in the situation of having to install Ubuntu without a CD-ROM, on a computer that has buggy or non-supported USB boot. From there, the easy way – I think – is booting from the network, since motherboard usually support PXE.
Booting from the network needs three things:
- A DHCP server for sending the address of the TFTP server and the pxelinux.cfg file to the DHCP client on the motherboard;
- A TFTP server setup to serve pxelinux.cfg and the operating system image;
- The operating system image.
I already have a DHCP server on my home router, and it runs DD-WRT. I had already enabled DNSmasq for DHCP, but I was missing the magic line in Services->Additional DNSMasq Options that tells the motherboard the address of the TFTP server:
DD-WRT can run a TFTP server, but that’s a bit complicated. I decided to install dnsmasq on my Ubuntu desktop, because dnsmasq provides, well, DNS, but also a TFTP server.
sudo apt-get install dnsmasq
It also provides a DHCP server, which will be a problem if connected to my home network because it will fight will my DD-WRT DHCP. So I need to disable that and only enable the TFTP server. I modified /etc/dnsmasq.conf to end up with everything commented-out except:
If fact, it already comes with the DHCP server disabled, because the ‘dhcp-range’ configuration is commented-out by default. I’m specifying the interface because I have more than one network card, and one of them was already busy (used by VirtualBox) and dnsmasq failed to start. Once you’re done with the changes, restart the daemon:
sudo /etc/init.d/dnsmasq restart
You could now try to boot, but the PXE client will complain that it either cannot find a file to boot from, or the mysterious « no boot filename received » message, depending on your firmware. The computer I’m installing has two network cards, and this added a variable to the debugging process, so I plugged both cards in my home switch.
Now this will never work without an operating system to boot. Ubuntu network boot images are available at:
Get the right netboot.tar.gz for your target machine, and decompress it in the TFTP server root (defined above in dnsmasq.conf):
sudo tar -xvzf netboot.tar.gz -C /var/ftpd
You should now be able to boot Ubuntu and proceed with the installation.
It didn’t work very well for me. The TFTP client reported it got a response, but then the computer froze. I checked the logs of the TFTP server for clues, and found that the client requested many files, and among them a vesamenu.c32. Knowing that VESA means graphical, my hypothesis was that the crappy video card isn’t supported, so I commented-out the corresponding line from pxelinux.cfg/default and fixed my problem!
As they say, your mileage may vary.
I won a licence for Structure101 at the last Code Retreat, watched introduction videos, and decided to give it a try. I looked for a decent Java project to run it on, and I picked one of the most starred projects on GitHub: storm.
First, Structure101 wants to know where the bytecode is, so I need to compile storm. I look for compile instructions and I find it contains Clojure sources. Since that should compile to bytecode, I take a chance and keep going.
I see storm has a project.clj in the top-level directory mentioning lein, so I install leiningen. The latest is version 2, but I need to use version 1, as per the project.clj and the error I get otherwise:
~/storm (master)? $ lein help ERROR: requires Leiningen 1.x but you are using 2.0.0-preview10
So I type lein compile, but it ends with a failure:
Compiling backtype.storm.messaging.zmq 1 [main] ERROR org.apache.zookeeper.server.NIOServerCnxn - Thread Thread[main,5,main] died java.lang.UnsatisfiedLinkError: no jzmq in java.library.path, compiling:(zmq.clj:1)
I find bin/install_zmq.sh and execute it, and get:
checking for ZeroMQ… no
checking zmq.h usability… no
checking zmq.h presence… no
checking for zmq.h… no
configure: error: cannot find zmq.h
It turns out the script does not check if commands failed and goes on with the rest. Before that error, zeromq complained about missing uuid-dev, and I `sudo apt-get install uuid-dev`.
This helps, but then jzmq wants to make org/zeromq/ZMQ.class but can’t find the classdist_noinst.stamp « target ». It turns out this is a known issue with a simple fix to do in src/Makefile.am. Finally `lein compile` completes.
Creating Structure101 project
I go back to Structure101 and create a new local project, picking Static classpath because I’ll point it to the generated `classes` directory. I choose the Logical breakout because I don’t really care how storm organizes its classes for the moment.
I choose the detailed granularity because the code base is not very big, and I guess I can reduce the amount of detail later if the view looks too cluttered.
For now I’ll hide external dependencies to concentrate on the storm codebase.
No excludes, no additional source files, no transformations.
It takes only few seconds to generate the summary.
It also says « The architecture for this project is not defined ». From the text following that note, I understand this is me that needs to define the desired architecture rules that must be obeyed, and which will be used to generate a violations report, possibly viewable in an IDE such as Eclipse or IntelliJ.
Structure: Exploring Tangles
In the summary, one big tangle that adds Excessive Structural Complexity (XS) is the `backtype.storm` package. Navigating to that package under the Structure tab, from the Package Explorer, I get this:
This is the Composition view. The dependency graph shows, at the top, that command, planner, and drpc packages depend on the tangle. At the bottom, the tangle depends on cluster, event, messaging, stats, nimbus, util, and ui. The `generated` package is even lower because it depends on nothing else.
Within the tangle, the first observation I make is that we see both the `backtype.storm` package as well as its subpackages. We can see that classes in `backtype.storm` refer to classes within sub-packages, and vice-versa. This discourages letting a package depend on its parent, where each level in the package hierarchy builts on its leafs.
The reverse of that – leafs depending on its parent – would mean that leafs couldn’t be refered to, other than by a package in a different root package. I see this can lead to interesting discussions, and how enforcing rules can help large teams build maintainable systems.
Back to the diagram, the `tuple` package is clearly a dependency of all other packages. The `utils` package could almost be a leaf, but it has a single reference to the `task` package (going up), although there are 13 references going the other way. This clearly is a smell, and I want to know what is that reference. Highlighting the arrow going up from utils to task, the Dependency breakout pane shows the culprit:
It shows that utils.ShellProcess takes a TopologyContext as a parameter. Navigating (with the context menu) to consumers of TopologyContext, I find that it is referenced 160 times from all over the place. This is often what happens with « context » classes: they tend to become a very important holder of state, and it is eventually passed to whoever needs access to any state. As a first step to untangle this, I could move ShellProcess out of utils into its own package, but I can’t find a way to do this in Structure101. This is most likely only possible in Restructure101.
Structure101 also has a standard Class Hierarchy viewer, to which I can navigate from the context menu of any class or interface represented in the dependency graph. For example, here’s `backtype.storm.spout.ISpout`:
I feel like I’m navigating classes in Eclipse, but with a bird’s-eye-view. I can see what call what, but instead of jumping from one method to the other, I get lists that interact with my selection in the graph. It feels natural, and lets me explore the code base.
I have been looking at a fairly big project, after all. It is all foreign code to me, so that does affect how fast I can learn to use Structure101 effectively. I’d be interested to use the tool for a new project, seeing the dependencies evolve and provide architectural feedback during development. This introspection would certainly help me understand how to write without tangles from the start.
I have not tried to define an architecture diagram, but I can see how defining a target architecture can be useful in identifying tiny steps to get toward the goal. Moving classes around is relatively easy with a good IDE, but if I was to reduce the dependencies on the TopologyContext class, for example, I would need to keep track of that over a long period.
Clojure generates many classes. This makes the experience of looking at class dependencies a bit difficult for that part of the code. However, the navigation and display shows packages first, hiding them unless they are part of a tangle.
Getting the most of the tool requires a bit of reading and trial and error. With experience, key observations would be faster to extract.
The Walkthrough guide refers to an older version of Structure101, and is difficult to follow. The user interface has changed, and there are dead links.
The right-click menu is broken, at least on the latest Ubuntu 12.10. The menu pops, but the mouse does not hightlight the items. Using the keyboard to navigate the contextual menu works, though.
I wish there were more keyboard bindings. Using arrows, enter, space, and context menu would help.
I won’t stop here. I’ll try Restructure101 to resolve problem I see in dependencies, in a « what if » mode of exploration. And the next step would be to use it on code I own and know well.
This is not something I would tell everybody to install, as the US$600 price tag is prohibitive for individuals – Restructure101 is even more expensive. The 30-day trial version can be useful for spot architectural work, but I’m glad I won a licence, as this will give me the opportunity to try it with various projects over a longer period of time.