This is continuation of my previous journal on working with F# in comparison to an existing C++ program and supporting Python systems. It ended on a still-prototype level system that was heavily OO, but used some of F#’s nicer type and syntax features to cut down boilerplate and organizational complexity. F# was easy enough to learn and use, but I was looking to push the system from a prototype to something more polished.

Functional Reactive Programming

A few weeks passed after petering out on the first attempt to work on the prototype. I had what I thought was a sane object structure, but I was getting bogged down building up the same domain model as before. I started looking for new tools to make the work go smoother. The success I had with relying on events to compose functionality was starting to break down. The fact that F# had first class events and good async support made this much easier to drop in and play with than C++ or Python. I realized that I was using more and more event handling code for a non-GUI app, so I decided to pursue that direction. I eventually absorbed a wealth of FRP knowledge and tools that looked like a great fit for my use case because it was a standard implementation of the patterns I was already performing ad-hoc. I quickly added the reactive extensions library (Rx) to my solution and started converting all events and interclass communication to use it for consistency. I was picturing this FRP framework to take on the role similar to dependency injection in an OO framework. It would be the routing glue that connected all the islands of functionality that was tied up in classes. All of my program’s inputs were bound to a custom observable to push data to whatever class had registered for it. Then there were intermediate pipelines that held most of the business logic about how the components interacted.

I quickly ran into a problem of organization and understanding of the pipeline. I had to break down much of my larger classes to expose events that could be consumed externally, which created small pieces of state encapsulated within each stage of the event pipeline. Changing the pipeline to expose a new piece of data or connect one stream to synchronize with another was very difficult. Instead of exposing clean connections between objects for data to flow, it showed me just how much entanglement the business rules dictated. I tried to simplify the model, but changing it too much into a stream format would mean that it wouldn’t be as understandable to the domain experts. I hesitated to throw away my current understanding of the business concepts just to embrace this methodology fully.

My partial adoption wasn’t a pleasant implementation experience. Just attempting to step through the procedure serially was more difficult because the events muddied up the call stack and hid where the supplied data came from. Reading the code also became more difficult, as the minimal ceremony that replaced the boilerplate code didn’t portray the structure of a program that was performing transforms over simple data series. While each supporting piece of code was now smaller, it was more difficult to see how it fit into the rest of the data flow picture. I had traded clarity for conciseness. The level of abstraction for the transformers was often greater than I needed. The FRP pieces themselves were easy to connect and use with a bit of reading, and they worked independently and offered many degrees of freedom. I wasn’t planning on managing hundreds of constantly reconfiguring streams of data. I wanted to manage two dozen streams that had maybe 5 configurations each with many transformation steps to syncornize, but few dynamic interactions. The number of configurations would be fixed, or at least very well specified up front. The additional flexibility in connecting data events to functions might have been helpful if I was just starting out and changing my data and control flow constantly, but I already had a working mental model of how the data should be processed and stored. Translating that structure into events and streams took time and serious mental effort to track of the many moving and interacting pieces. If I was lost before without big objects interacting in a web, I was starting to feel even more confused by thinking in streams.

After I made backwards progress by breaking functionality that was easy to understand and working, I began to realize that FRP might not be a great fit for my application. I could see how it would have worked well for a GUI or a domain that had more independant data, but since the business logic relied on conditional transforms and synchronizations, the logic to coordinate it all was a mess. FRP seemed much better suited to modeling complex control networks than for multi-channel data streaming. It’s possible that I was going about this process all wrong and there was a key technique that could have made this work better. The code and events did compose more cleanly than my large classes, so I could feel I was onto something, but I wasn’t making progress. Breaking up the pieces did help me learn more compositional patterns, but most of the work turned into writing what felt like overly complicated or very simplified LINQ. It was hard to get an even distribution of complexity and still maintain a layout that was close to the business model.

Stepping Back

I only spent a few days working on this after a few weeks of reading and I didn’t like the direction it was headed. I needed to step back from trying new things and focus on other supporting aspects of the language. I had learned quite a bit about the practical details of using F#, but since I had been distracted by the glamor of FRP I was hiding my gains.