Evaluating Structure101 with StormPublié : décembre 20, 2012
I won a licence for Structure101 at the last Code Retreat, watched introduction videos, and decided to give it a try. I looked for a decent Java project to run it on, and I picked one of the most starred projects on GitHub: storm.
First, Structure101 wants to know where the bytecode is, so I need to compile storm. I look for compile instructions and I find it contains Clojure sources. Since that should compile to bytecode, I take a chance and keep going.
I see storm has a project.clj in the top-level directory mentioning lein, so I install leiningen. The latest is version 2, but I need to use version 1, as per the project.clj and the error I get otherwise:
~/storm (master)? $ lein help ERROR: requires Leiningen 1.x but you are using 2.0.0-preview10
So I type lein compile, but it ends with a failure:
Compiling backtype.storm.messaging.zmq 1 [main] ERROR org.apache.zookeeper.server.NIOServerCnxn - Thread Thread[main,5,main] died java.lang.UnsatisfiedLinkError: no jzmq in java.library.path, compiling:(zmq.clj:1)
I find bin/install_zmq.sh and execute it, and get:
checking for ZeroMQ… no
checking zmq.h usability… no
checking zmq.h presence… no
checking for zmq.h… no
configure: error: cannot find zmq.h
It turns out the script does not check if commands failed and goes on with the rest. Before that error, zeromq complained about missing uuid-dev, and I `sudo apt-get install uuid-dev`.
This helps, but then jzmq wants to make org/zeromq/ZMQ.class but can’t find the classdist_noinst.stamp « target ». It turns out this is a known issue with a simple fix to do in src/Makefile.am. Finally `lein compile` completes.
Creating Structure101 project
I go back to Structure101 and create a new local project, picking Static classpath because I’ll point it to the generated `classes` directory. I choose the Logical breakout because I don’t really care how storm organizes its classes for the moment.
I choose the detailed granularity because the code base is not very big, and I guess I can reduce the amount of detail later if the view looks too cluttered.
For now I’ll hide external dependencies to concentrate on the storm codebase.
No excludes, no additional source files, no transformations.
It takes only few seconds to generate the summary.
It also says « The architecture for this project is not defined ». From the text following that note, I understand this is me that needs to define the desired architecture rules that must be obeyed, and which will be used to generate a violations report, possibly viewable in an IDE such as Eclipse or IntelliJ.
Structure: Exploring Tangles
In the summary, one big tangle that adds Excessive Structural Complexity (XS) is the `backtype.storm` package. Navigating to that package under the Structure tab, from the Package Explorer, I get this:
This is the Composition view. The dependency graph shows, at the top, that command, planner, and drpc packages depend on the tangle. At the bottom, the tangle depends on cluster, event, messaging, stats, nimbus, util, and ui. The `generated` package is even lower because it depends on nothing else.
Within the tangle, the first observation I make is that we see both the `backtype.storm` package as well as its subpackages. We can see that classes in `backtype.storm` refer to classes within sub-packages, and vice-versa. This discourages letting a package depend on its parent, where each level in the package hierarchy builts on its leafs.
The reverse of that – leafs depending on its parent – would mean that leafs couldn’t be refered to, other than by a package in a different root package. I see this can lead to interesting discussions, and how enforcing rules can help large teams build maintainable systems.
Back to the diagram, the `tuple` package is clearly a dependency of all other packages. The `utils` package could almost be a leaf, but it has a single reference to the `task` package (going up), although there are 13 references going the other way. This clearly is a smell, and I want to know what is that reference. Highlighting the arrow going up from utils to task, the Dependency breakout pane shows the culprit:
It shows that utils.ShellProcess takes a TopologyContext as a parameter. Navigating (with the context menu) to consumers of TopologyContext, I find that it is referenced 160 times from all over the place. This is often what happens with « context » classes: they tend to become a very important holder of state, and it is eventually passed to whoever needs access to any state. As a first step to untangle this, I could move ShellProcess out of utils into its own package, but I can’t find a way to do this in Structure101. This is most likely only possible in Restructure101.
Structure101 also has a standard Class Hierarchy viewer, to which I can navigate from the context menu of any class or interface represented in the dependency graph. For example, here’s `backtype.storm.spout.ISpout`:
I feel like I’m navigating classes in Eclipse, but with a bird’s-eye-view. I can see what call what, but instead of jumping from one method to the other, I get lists that interact with my selection in the graph. It feels natural, and lets me explore the code base.
I have been looking at a fairly big project, after all. It is all foreign code to me, so that does affect how fast I can learn to use Structure101 effectively. I’d be interested to use the tool for a new project, seeing the dependencies evolve and provide architectural feedback during development. This introspection would certainly help me understand how to write without tangles from the start.
I have not tried to define an architecture diagram, but I can see how defining a target architecture can be useful in identifying tiny steps to get toward the goal. Moving classes around is relatively easy with a good IDE, but if I was to reduce the dependencies on the TopologyContext class, for example, I would need to keep track of that over a long period.
Clojure generates many classes. This makes the experience of looking at class dependencies a bit difficult for that part of the code. However, the navigation and display shows packages first, hiding them unless they are part of a tangle.
Getting the most of the tool requires a bit of reading and trial and error. With experience, key observations would be faster to extract.
The Walkthrough guide refers to an older version of Structure101, and is difficult to follow. The user interface has changed, and there are dead links.
The right-click menu is broken, at least on the latest Ubuntu 12.10. The menu pops, but the mouse does not hightlight the items. Using the keyboard to navigate the contextual menu works, though.
I wish there were more keyboard bindings. Using arrows, enter, space, and context menu would help.
I won’t stop here. I’ll try Restructure101 to resolve problem I see in dependencies, in a « what if » mode of exploration. And the next step would be to use it on code I own and know well.
This is not something I would tell everybody to install, as the US$600 price tag is prohibitive for individuals – Restructure101 is even more expensive. The 30-day trial version can be useful for spot architectural work, but I’m glad I won a licence, as this will give me the opportunity to try it with various projects over a longer period of time.