Monkeying with the code

A couple of people have already covered this already, specifically Bobby Johnson, Matt Hinze, and Zachariah Young. I figure I should say something on it anyway.

I’ve adopted a project from Jeremy Miller that I think has the potential to be a really useful tool. It’s called Fluent NHibernate, and it’s primarily a fluent API for mapping classes with NHibernate.

We’re all well aware how awesome NHibernate is, but I think we all also have a bit of a dislike for the amount of XML you need to write to get your classes mapped; not only that, but also how the mappings are distinctly separate from the rest of your application. They’re often neglected and untested. One of the core tenets of the project is that we need a more succinct, readable, and testable way of writing your mappings.

The API

Take the following simple hbm file:

<?xml version="1.0" encoding="utf-8" ?>
<hibernate-mapping xmlns="urn:nhibernate-mapping-2.2"
    namespace="Eg" assembly="Eg">

    <class name="Customer" table="Customers">
        <id name="ID">
            <generator class="identity" />
        </id>

        <property name="Name" />
        <property name="Credit" />

        <bag name="Products" table="Products">
            <key column="CustomerID"/>
            <one-to-many class="Eg.Product, Eg"/>
        </bag>

        <component name="Address" class="Eg.Address, Eg">
            <property name="AddressLine1" />
            <property name="AddressLine2" />
            <property name="CityName" />
            <property name="CountryName" />
        </component>
    </class>
</hibernate-mapping>

Then compare it to the same mapping, created using the fluent API:

public CustomerMap : ClassMap<Customer>
{
  public CustomerMap()
  {
    Id(x => x.ID);
    Map(x => x.Name);
    Map(x => x.Credit);
    HasMany<Product>(x => x.Products)
      .AsBag();
    Component<Address>(x => x.Address, m =>
    {
        m.Map(x => x.AddressLine1);
        m.Map(x => x.AddressLine2);
        m.Map(x => x.CityName);
        m.Map(x => x.CountryName);
    });
  }
}

Firstly, you’ll note that there is a marginal reduction in lines of code, but that’s not what we’re particularly striving for. Instead we’re intent on reducing the verbosity and noise of the code. This manifests itself in a convention over configuration design for the API, where we choose the most common setups and use those as the default. For example with the id element in the hbm file, you’re required to specify what the generator type is; however, in our fluent API we check the type of your identity property and decide what generator we should use. Int’s and longs default to identity, while GUIDs use the guid.comb generator. You can change these explicitly, but when you are using the default, it greatly reduces the verbosity of your mapping.

Testability

Another one of our goals is to make your mappings more robust. I imagine most people have had the problem where you’ve renamed a property and not updated the mapping file; due to there being no compile time validation, the only way to catch these mistakes are at run time (hopefully you had tests to cover that!). With the way our API is designed, you use the actual properties on your classes to create the mapping, so there’s nothing to forget. If you rename a property, your IDE will either rename the property in the mapping, or fail at compilation.

We also want to help you verify that your mappings are set up properly, not just syntactically valid. So to make your integration tests a bit easier, we’re providing an API for testing your mappings.

[Test]
public void VerifyCustomerSaves()
{
    new PersistenceSpecification<Customer>()
        .CheckProperty(x => x.Name, "James Gregory")
        .CheckProperty(x => x.Age, 22)
        .VerifyTheMappings();
}

Behind the scenes the PersistenceSpecification creates an instance of your entity, then populates it with the values you specify through the CheckProperty method. This entity is then saved to the database, then reloaded through a separate connection. The returned entity is then compared to the one originally saved, and any differences fail the test. It’s a fairly standard integration test, except we’ve taken the time to write all the wiring up that needs to be done, so you don’t have to.

The Framework

We’re working towards our first official release, which will have a fairly solid implementation of the API. Once that’s out in the wild, we’re going to focus on our Framework.

Our framework is a layer that sits on-top of the API to provide an even better experience. We’re looking to integrate with your favorite container, which will reduce the code you need to write to integrate NHibernate into your system. Then we’re going to tackle extensible conventions, which will allow you to specify your own implied conventions for your application. For example, if you’re always going to call your identifier “ID”, then why should you have to specify it every time? You shouldn’t!

Development is progressing at a nice pace, and I expect we’ll be able to get our first release out within the next few weeks. The testing API hasn’t been kept quite as up to date as the main API, but we’re working on that too. It’s open-source, so suggestions and patches are welcome.

It’s been a long time since I’ve written anything about BooLangStudio. Let me assure you that the development is still very much underway, and we’re steadily working towards our first release.

To fill you in a bit on what’s been going on. Jeffery Olson has completely rewritten the syntax highlighting to use the more flexible Boo.Pegs lexer, which allows us to overcome some of the obstacles we were facing with the traditional Boo lexer. There were limitations to what we could do with the traditional lexer, without modifying the Boo source. This was leading us down a road we weren’t keen on, one of maintaining a fork of the Boo source that contained our specific changes. With the move to the Boo.Pegs lexer, we’re free of this scenario!

Justin Chase has been tackling various issues all over the place. Several related to the build process, specifically how the Boo binaries are located on your machine, which has been a bit of a problem when there’s multiple developers working on a project whom both have Boo installed in a different location. He’s also been spiffing up the interface by adding some much needed icons to the projects.

Torkel Ödegaard has been working various issues throughout the project. He’s created a properties dialog for the project which allows you to alter the usual settings for your project, assembly name, default namespace etc… He also managed to get debugging working, which is awesome.

Then there’s me, I’ve been working on the intellisense capabilities. We’ve had intellisense support from day one, but it’s never been very extensive. For a long time it only worked on the current file you had open.

The entire solution is now processed behind-the-scenes to provide intellisense. Import statements are now recognised and imported types appear in the local scope intellisense. Any assemblies or projects that are referenced have their top-level types and namespaces included in the intellisense too.

Additionally I’ve been working on improving the method suggestion logic, which was initially very flakey. It boils down to being able to transform the current line into the desired type, which can be difficult in certain cases (StringBuilder().Append("Hello world").ToString(). for example).

I believe that about covers what we’ve been upto lately. Our biggest problem currently is that most of these features are implemented in separate branches, and haven’t yet been merged into our central repository. We’re working on this, and I reckon once we’ve done that we’ll be looking at doing a release.

Ever get the feeling that you’ve missed out on something big? I never got to see Star Wars at the cinema*. I missed out on the start of computers, the ASM, the BASIC hacking and what-not. I discovered most of my musical taste way after the bands were gone or going, so I never got to see them in show.

None of those are my fault, of course, because they were all before my time; in fact, most were at least a decade before me. That still doesn’t stop me from ocasionally getting the feeling that I missed out on something big.

One more. One that’s pertient to our industry: Smalltalk.

Smalltalk is one of those languages, possibly the most prominent example, that has been idolised over time. It’s often touted as having an excellent programmer experience, properly object-oriented, the source of refactoring, great tools, and a solid virtual machine behind it. It’s one of those languages that people are proud to say they once worked with it. A language that people nosalgicly look back on. It’s also got an air of betamax about it (yeah, I missed that one too), people were sure it was better, but it still didn’t become mainstream.

We Smalltalkers used to think the advantages of our language were so significant that it would take over the world. We had a huge productivity advantage over C coders. Then C++ came along and gave C coders just enough to let them improve their productivity and their ability to write larger more complex systems. It still wasn’t as good as Smalltalk, but it was better than C, and much more accessible to most programmers than Smalltalk.

-Josh Susser via Raganwald

So where am I going with this? Just like I can make myself feel a bit better by watching Star Wars on a large TV, or listening to the Rolling Stones live albums. I’d like to see what all the fuss was about with Smalltalk, by giving it a go.

I’m not sure where to start on this, or how long it’s going to take, or even when I’m going to do it; however, one day in the not too distant future I’m going to do some programming with Smalltalk.

* Although I actually did get to see A New Hope when they did the re-release special edition, which made up for it.

Quick update on the Boo plugin I mentioned previously. Jeff Olson announced the BooLangStudio project a couple of days after I announced mine, and we’ve decided to join forces. I’ve begun merging our codebases, and we’re looking to get a release out in the not too distant future.

Our combined code-bases have now got syntax highlighting, intellisense, single file and project support. Not bad going.

We’ve now got a nice little team going as well, which is a big improvement from what was the one man band of my self. One of the contributors, Torkel Ödegaard, has written a nice overview of our progress and processes as-well.

What do we want? A Boo plug-in for Visual Studio. When do we want it? Sooner, rather than later.

The initial aim of this plug-in is to provide a “good enough” experience when using Boo files within other projects. I’m not currently focussing on doing full-scale development with Boo; so the target is DSLs and Binsor. There won’t be any assemblies created, or anything else that would affect your project. The only difference between how you work now would be that you could write Boo using Visual Studio, with syntax highlighting and intellisense.

That’s for the initial version anyway.

I’ve found that creating a plugin for Visual Studio isn’t that difficult, in-fact that’s an understatement, it’s stupidly easy. Most of the plugin architecture is successfully created using the project wizard, so there’s very little I needed to do from that point of view. The tricky part is getting Boo and Visual Studio to communicate.

I started by spending some time looking at the IronPython Studio implementation, the Boo implementation, abstract syntax trees, antlr, and after getting my head around what I needed to do, I set out on my adventure.

So what’s the result? A very basic (and flakey) implementation of both syntax highlighting and intellisense.

The highlighter isn’t perfect yet. It suffers from slowdown when you’re writing an unclosed string, and it can’t handle multi-line statements yet (think comment blocks).

The intellisense is pretty basic, it allows namespace navigation, and type methods, but there isn’t much in the way of constraints (you can see instance methods against types, for example).

There’s a lot of work to be done, but it’s fun and interesting.

Targets

In the near future I’d like to get the following done:

• Make intellisense work with everything in the local scope
• Make intellisense parse import statements
• Make syntax-highlighting correctly handle multi-line blocks
• Make syntax-highlighting highlight types (like C# does)

Then at some point tackle these:

• Get Resharper involved
• Refactoring support

Hopefully I’ll progress on these and report back in a future post.

Code

The code is poor, it’s a mess and probably all wrong; however, you can get it in the usual place. However, I haven’t provided a direct download because it isn’t really in a state to be distributed yet.

The source is accessible from Subversion at: http://jagregory.googlecode.com/svn/trunk/BooPlugin/ (using user jagregory-read-only)

The current system I’m working with has a setup whereby the model (data access objects) are being used by the web-services for delivery; this means that the consumers of the web-services are directly tied to our inner implementation of our data access code. On-top of that, it’s old full of bad conventions, and is in need of refactoring.

This is my account of how I freed the model from the web-service.

public class vehicle_info
{
    public int vehicle_id;
    public string vehicle_name;
    public string vehicle_manufacturer;
    public DateTime vehicle_manufactured_date;
    public int vehicle_number_produced;
}

That’s the existing model. You can see that these have some nasty naming conventions that I just can’t live with anymore.

A large problem with pushing your model out to the web-service consumers is that I can’t refactor the classes without breaking their implementations; if I rename a field in my model, the consumers will have to update their code to deal with it. This is an issue because not all the consumers are able to update their code. So we’re stuck with how the classes were built years ago, with awful naming conventions, and no ability to change.

What’s the ideal setup then? It’s my opinion that we should be using specialised DTOs (Data Transfer Objects) for this purpose, they’ll exist solely to be the transport mechanism for the web-service. With DTOs you’re separating what your service pushes out from what you use internally, which allows you to refactor your internal design without any problems with integration.

public class VehicleWebServiceDto
{
    public int vehicle_id;
    public string vehicle_name;
    public string vehicle_manufacturer;
    public DateTime vehicle_manufactured_date;
    public int vehicle_number_produced;
}

There’s an issue with using DTOs in an existing web-service environment, it’s unlikely your DTOs will be named the same as your original objects (if they do, you’re fine). We can’t just swap it out with a DTO, because the signatures won’t match in the WSDL.

When we change the web-service to return those DTOs instead of the original classes, we end up with this mismatch in the output:

<vehicle_info>
  <vehicle_id>int</vehicle_id>
  <vehicle_name>string</vehicle_name>
	...

<VehicleWebServiceDto>
  <vehicle_id>int</vehicle_id>
  <vehicle_name>string</vehicle_name>
	...

How can we get around this? Make the signatures match.

You can control the way objects are serialized by the web-service through the serialization attributes; namely the XmlRoot and SoapType attributes are the ones you’re looking for, these allow you to override what name is output in the xml for your class. Slap on those attributes, one for the standard HTTP usage and the other for SOAP, and give them your old class name; then as far as the consumers of the webservice can see, you’re giving them the same old classes. Now you have a separate DTO from your domain layer, but without breaking the schema.

These are the updated DTO objects:

[XmlRoot("vehicle_info"), SoapType("vehicle_info")]
public class VehicleWebServiceDto
{
    public int vehicle_id;
    public string vehicle_name;
    public string vehicle_manufacturer;
    public DateTime vehicle_manufactured_date;
    public int vehicle_number_produced;
}

Which now produce this xml:

<vehicle_info>
  <vehicle_id>int</vehicle_id>
  <vehicle_name>string</vehicle_name>
	...

We are now free to refactor our code as much as we like, without affecting the output of the web-service.

public class Vehicle
{
    public int ID { get; set; }
    public string Name { get; set; }
    public ManufacturingDetails ManufacturingDetails { get; set; }
}

public class ManufacturingDetails
{
    public Manufacturer Manufacturer { get; set; }
    public DateTime ManufacturedDate { get; set; }
    public int NumberProduced { get; set; }
}

public class Manufacturer
{
    public int ID { get; set; }
    public string Name { get; set; }
}

Magic.

Making a Visual Studio 2008 plugin isn’t as hard as it sounds.

Making one that works is a bit more difficult.

Syntax highlighting is fairly straight forward, just run a tokenizer over the text and colour code each token appropriately. It’s the code sense that is causing me to go slightly mad. To determine what methods need to be listed when the intellisesne pops up, you need to parse the code that’s in the document, then resolve any classes and references in there to build up a collection of methods to display. It gets more tricky when the document isn’t valid, because you can’t compile it. That’s where I am now, I need to figure out some way of partially compiling the document, or maybe using an AST walker.

I just posted about how if you’re looking for a solution to a problem that hasn’t already been solved, then you’re probably doing something wrong. Well this is my case of it.

I quite dislike mapping DTOs to entities, it’s a pain, but mostly tedious and tiresome rather than difficult. I decided to try to ease things by creating a library that would resolve entity instances to their DTO counterparts.

My requirements were few but I was determined not to violate any of them.

1. Refactoring friendly. No strings for property names, changing names should give compiler errors.
2. Must simplify code.
3. Must improve maintainability.

First attempt: Explicit mapping

var mapper = new DtoMapper<Customer, CustomerDto>();

mapper.Pair(mapper.Entity.Name, mapper.Dto.CustomerName);
mapper.Pair(mapper.Entity.Address, mapper.Dto.CustomerAddress);

// ... elsewhere ...

CustomerDto dto = mapper.Transform(customer);

With a clever bit of DynamicProxy usage, this implementation successfully mapped properties on an entity to a DTO. I believed it was reasonably clear, but having to use mapper.Entity was a bit obtuse. Dealing with properties on instances without an instance is tricky, especially if you want to avoid using strings.

The explicit mapping is very refactoring friendly. I could rename a property without breaking the mapping, so requirement 1 was satisfied.

var dto = new CustomerDto();

dto.CustomerName = customer.Name;
dto.CustomerAddress = customer.Address;

return dto;

As the above code demonstrates, this implementation isn’t actually any simpler than just doing simple assignments, it’s in-fact more complicated because of the overhead of understanding what the mapper is. This simple assignment method is also refactoring friendly.

So with requirement 2 failed, and requirement 3 no different to doing it manually, it was time to move on.

Second attempt: Implicit mapping

var mapper = new DtoMapper();

mapper.CreateImplicitMap<Customer, CustomerDto>();

// ... elsewhere ...

CustomerDto dto = mapper.Transform(customer);

This is my favorite implementation, it’s clean and smart; however, it’s also useless.

It did the same as the first example, but implicitly mapped any properties that have the same name together. This is fine, but it would ignore anything that don’t have the same names. I could’ve implemented some logic for guessing names, but that would just be asking for trouble.

Tragically this implementation wasn’t that refactoring friendly; you can rename properties, but it would silently stop mapping them unless you renamed it’s partner too. That’s pretty dangerous stuff. Requirement 1 failed.

It does produce less code than the first implementation, and the simple assignment method, so 2 and 3 are covered.

Third attempt: Attributes

public class CustomerDto
{
	[DtoPartner(typeof(Customer), "Name")]
	public string CustomerName { get; set; }

	[DtoPartner(typeof(Customer), "Address")]
	public string CustomerAddress { get; set; }
}

I could live with this implementation, it’s not as clean as the implicit method, but it is still quite clear.

Unfortunately, it fails in the refactoring test. You can’t rename a property on Customer without it breaking the mapping, because the property names are strings. Requirement 1 failed.

You could smooth over this with an inspection unit test, which checks the strings against their types to see if the property exists, but that smells, it’s not a very good library if you have to verify it even works. I could’ve also created a static class to represent the Customer instance properties, but that’s more noise (you’d need 3 classes, instead of just 2); a pre-build step (ala SubSonic) came to mind, but that’s entering into the realm of diminished returns.

Conclusion?

Sometimes the obvious way is the best way. Old fashioned may be old, but that doesn’t make it wrong. Sometimes a cigar is just a cigar.

If you have a really good idea, and nobody has done it before, chances are there’s a reason for that. It could be that it was technically impossible last time looked at it, but more than likely it’s because you’re trying to solve something that is only present because you’re doing something wrong.

Consider alternatives always before embarking on writing something yourself. In an ever maturing software ecosystem, and the constant catch-up .Net plays with Java, if a solution to your problem was really needed, somebody should have attempted it before.

That’s not to say don’t innovate, but don’t recreate either, and certainly don’t provide solutions to non-existent problems.

This article serves as an introduction to the concept of Dependency Injection, and why you’d want to use it. It is not a getting started guide for using containers. If you’re interested in those, my personal preference is Castle Windsor and you can find their getting started guide here.

What are dependencies? Also referred to as couplings, dependencies are other modules that one module requires to fulfill it’s purpose.

Are dependencies bad? No, of course they’re not, otherwise we wouldn’t be able to create anything. However, highly coupled code can cause a lot of problems for your application.

If your code requires knowledge of how a dependency works, then your code is highly coupled. If your code is tied explicitly to an implementation, then your code is also highly coupled.

Take the following example:

public class ProductUpdater
{
	public void StoreChanges(Product product)
	{
	  SqlDataStore dataStore = new SqlDataStore();

	  dataStore.CreateConnection();
	  dataStore.OpenConnection();

	  dataStore.Update(product);

	  dataStore.CloseConnection();
	}
}

The above example is highly coupled to the SqlDataStore. Firstly, it directly creates an instance, which means there’s no way for us to replace that instance if we need to (I’ll come to why you’d want to do that in a bit). Secondly, it relies on a great deal of knowledge of SqlDataStore’s implementation. In this code we can see that you need to create a connection and open it before you can update the record; that’s quite a bit of implementation knowledge. If the SqlDataStore was to change so that the OpenConnection method created a connection if one didn’t already exist, then we’d need to change every caller of that code to remove the CreateConnection call; in large system situations like that can quickly lead to a fear of change and refactoring.

I mentioned directly creating an instance stops us from replacing it if need be. Well when would you actually want to do this? For those unfamiliar with unit testing, you probably haven’t encountered test doubles; there are different types of test doubles, but for the purposes of this example they’re interchangeable.

A test double serves as a swap-in replacement for one of your dependencies. These allow you to execute a piece of code under test, without having to worry about whether things are being put in your database, or e-mails sent for example. If your code is creating instances within methods, those instances cannot be replaced by a test double. Without that ability, testing becomes considerably more difficult.

Tightly tying your code to an instance of a class reduces the flexibility and reuse of your code. Take the above example, that same code could be used to update the product in a cache instead of the database; similarly, you could use an in-memory storage instead of a database for when you’re running in a test or demo environment. If only your method wasn’t so tightly tied to the implementation.

This is solved by using Dependency Injection, which is a part of the and Inversion of Control.

[DIP] seeks to “invert” the conventional notion that high level modules in software should depend upon the lower level modules. The principle states that high level or low level modules should not depend upon each other, instead they should depend upon abstractions. — Wikipedia

Essentially, dependency injection allows you to stop instantiating your dependencies. Instead they’re “injected” into your object when it is instantiated itself. This allows the dependencies to be swapped out like we mentioned above.

So taking the original example, here’s a version of it that’s been updated to use dependency injection.

public class ProductUpdater
{
	private SqlDataStore dataStore;

	public ProductUpdater(SqlDataStore dataStore)
	{
		this.dataStore = dataStore;
	}

	public void StoreChanges(Product product)
	{
	  dataStore.CreateConnection();
	  dataStore.OpenConnection();

	  dataStore.Update(product);

	  dataStore.CloseConnection();
	}
}

The above implementation now allows us to create a test double, and replace the SqlDataStore in our tests. As I mentioned before, we could now easily push in an in-memory implementation without any changes to the code required.

We can take this further though, because we’re still tied to a concrete class. Lets make SqlDataStore implement an interface, so we can create other implementations.

public interface IDataStore
{
	void CreateConnection();
	void OpenConnection();
	void Update(Product product);
	void CloseConnection();
}

public class ProductUpdater
{
	private IDataStore dataStore;

	public ProductUpdater(IDataStore dataStore)
	{
		this.dataStore = dataStore;
	}

	public void StoreChanges(Product product)
	{
	  dataStore.CreateConnection();
	  dataStore.OpenConnection();

	  dataStore.Update(product);

	  dataStore.CloseConnection();
	}
}

Now our example is no longer specifically tied to a SqlDataStore, so we could quite easily pass it a FileSystemDataStore, or an InMemoryDataStore, or anything else that implements IDataStore. All that without having to touch a single line within the ProductUpdater.

That’s the power of dependency injection, and why you should stop hard-coding your dependencies.