It strikes me we already have all of the necessary technology to come up with a viable solution: I’ve worked with most of them! Let’s have a look what this might look like.

How It Would Work

Let’s say you’ve misplaced you pen: you whip out the portable device of your choice (in my case, my iPod Touch), launch the search app and type “pen”. This would either access a local index over the network, or an index on the web, and would start looking where an item that matches “pen” was last seen. Say you have seven pens at your home: one in the bedroom, three in your home office, two in the living room (of which one in the couch, between the cushions) and one in the laundry room.

In order for the search service to know where your pen is, the amount of data you need is surprisingly small: it needs to know where you are, what a pen is, which pens you might be looking for and where those pens were last “seen”. How would we go about getting this information?

Let’s start by looking at how to know where the pens were last seen: in order for a computer to “see” anything, it needs a sensor of some kind. That sensor, in our case, needs to provide enough information for the computer to know where it is and what it has observed. If we look at solutions for similar problems (always a good way to start an analysis), we might take a look at EPCIS (Electronic Product Code Information System): it is a system based on event captures in which sensors among the supply chain generate events that are stored in an EPCIS repository where they can later be analyzed. EPCIS events consist of several fields, including the EPC itself (an electronic product code), an action, a disposition and a read point. Each of these is identified by a URN that might look a bit like this: urn:epc:id:sgtin:1234567.123456.123 (which would be an EPC). Some of the information is inherent to the scanner (e.g. the read point) or its context (action and disposition), but one of the bits of information is part of the product: the EPC.

An EPC contains a lot of information, some of which we don’t need for our purposes: an SGTIN, for example, contains a serial number and a GTIN, which is a Global Trade Identification Number. The GTIN contains a Company Prefix and an Item Reference, which identifies the company that made (or imported, or distributed) the pen and the type of item it is (a pen). You can find the same information in a barcode. For example: 1-88874-84006-6 is a UPC-12 barcode that contains a company prefix 88874, identifying EMF Tea & Wares, and an item reference 84006, identifying a 120g container of “Thé au Jasmin”. Now, the way that EPCs are usually read is either by scanning a barcode that contains the EPC (usually a data matrix barcode), or by reading an RFID tag. RFID tags are relatively expensive – sufficiently so to hamper their rapid adoption for serialization (for supply line security) in the pharmaceutical industry – so we shouldn’t expect pens to be outfitted with RFID tags any time soon. However, while RFID tags are relatively expensive for production purposes, they are relatively inexpensive for retail purposes: a tag costs a few cents. RFID tags are also fairly small: they can be embedded into a label you’d find on a bottle of pills, for example. A typical tag would be about a 2 cm wide square, most of which is the tag’s antenna. It would be feasible, therefore, to tag objects you want to be able to search for with a stick-on RFID tag: you could basically put the tag inside the pen, rolled up, provided the pen is mostly made of plastic (or the tag could be embedded in a label that you stick to the pen).

RFID has the advantage of being “transparent” in two ways: because it works with radio waves, most objects are transparent to it (as are you, mostly), so your pen can be in your pocket and still be visible the the RFID reader. The other way it is transparent is that once you’ve stuck the tag on your object, you don’t have to think about it anymore: when you carry it around, an RFID reader will see the tag and generate an appropriate event.

This takes care of half of our first problem: items that already have an RFID tag that has a GTIN or (better yet) an SGTIN in it are duly identified and are now visible to RFID readers. The ones you tag, you have to associate with the object, so you can imagine having an RFID reader with your computer and, when you tag an object, scanning it and answering a few questions about the object (such as what it is). This begs the question, though: how do we get those tags to tell the readers where they are?

The answer is: we don’t. The tags I’m thinking of are passive tasks, so they wouldn’t normally be very chatty: they only talk when talked to (and get the energy to talk from the antenna, so there’s no batteries required). Antennas would be strategically placed around the house, where it makes sense. In my case, I’d put one at the top of the stairs, at each doorpost (on the inside of each room) etc. That way, an tagged object in my house would be “seen” by those readers at those strategic places. It would be practicallty impossible for my pen to end up in the laundry room without going, one morning, from the master bedroom down the stairs and through the door of the laundry room. Even if one of the readers doesn’t pick it up (e.g. the one in the laundry room) I can follow its trajectory down the stairs and have a good guess where it might end up if, say, the laundry basket took the same route at the same time.

Security

So, we now have a basic solution. I think the use-cases are obvious (they are to me, at least). What about abuse cases? Say I tagged a diamond necklace, which I would happen to have laying about the house. A thief coming into the house and whipping out his iPad could do a quick inventory of the place, know exactly where everything is and clean me out in no time, right?

That’s what you need security for: you don’t want any-one to have an idea of the inventory of your home, so if at all possible, the information should stay in your home – which means this becomes a localized solution. The database and software could easily be housed in a little box that could be installed next to your home router, and to which the various RFID readers would send their info through wireless LAN or through a home-area network (they wouldn’t have to be on-line all the time, either: battery-powered RFID readers would only need a simple motion sensor to turn themselves on, perform a reading, send the result and go back to sleep). The other bit of security – not allowing the burglar to read the data – is a question of securing your network and putting a password on the server. Of course, the readings being sent would have to be secured as well: both sides would have an X.509 certificate identifying themselves and thus allowing them to communicate.

Roaming

So, let’s say all of your friends have tagged their iPods, iPads, iMacs, laptops, pens, secret diaries, jewelery, family heirlooms, etc. as have you. You go to a party at one of your friends’ and fall asleep on the couch. When you wake up, you want to check the time on your tagged thingamajig and find it missing. What do you do?

Well, because your friend conveniently places a reader at the front door, his system picked up a “foreign object” belonging to user ID 12345 (part of the tag you put on it). Your friend whips out his Android phone, fires up the app and instructs it to tell it where anything belonging to user 12345 is. Most of it, being on your person, is reported as being in the living room but, somehow, your thingamajig left a few hours ago. At about the same time, a whole bunch of stuff belonging to user 23456 left as well. User 23456 being your girlfriend, you rest assured and hope she isn’t too mad at you for falling asleep on the couch (while she was talking to you).

Conclusion

Our time of searching for stuff should be over! If some-one would be willing to come up with the venture capital, I’ll be happy to make a product out of this idea. I already know everything I need to know to get the ball rolling and I can come up with a pretty good sketch of what both the software and hardware should look like pretty quickly.

                          

OK, but given by whom?

I love a good debate, and I am open to one: I have a few opinions to offer to start one off, too!

1. The purpose of science is not to ask “why?”, but rather to ask “how?”
   In fact, science cannot answer the question of “why?” because that question is, by its nature, part of the realm of metaphysics and philosophy rather than that of science. Even Kant with his “Kritik der reinen Vernunft” (Critique on pure reason) cannot get around the “every event must have a cause” logic of his predecessors and, as far as I know, no-one has.
2. If something is to be given, it must be given by someone (or something).
   In fact, “given the existence of gravity” doesn’t explain much other than the shape and form of the universe. The eleven dimensions proposed by Hawking aren’t new to his new book and may be explained by the existence of gravity – I haven’t read the book yet – but that hardly explains gravity.

Opinions?

                          

One thing you may have noticed when looking at the code of our abstract factory, is that the base classes (interfaces) of each of our abstract objects don’t have much to tell their users: there are hardly any accessors or mutators to be found. This is an attribute of encapsulation called opacity and in this installment, we’ll explore its advantages and disadvantages.

Let’s look at the Mechanism class: the base class says very little about the mechanism itself: it doesn’t even allow you to get its name. It currently only allows you to get credentials from the current mechanism and will eventually serve as the primary interface to interact with the mechanism. It will not, however, allow you any knowledge as to what that mechanism is. That knowledge, once it is handed to the MechanismFactory, is lost (unless you keep it around elsewhere).

The Credentials class is arguably even worse: it doesn’t allow you to know anything about the credentials and is effectively an opaque handle to the credentials behind them. This will remain true for the most part throughout the development of these libaries.

The opacity of both of these classes illustrates a general design principle that has definite advantages, but also some drawbacks. We will focus only on the opacity-attribute of this principle – the principle being encapsulation. The other attributes of the design principle of encapsulation will be handled at another time.

The general effect of opacity in design

One of the phrases that has become well-known for reasons outside of software design is the reply “that is on a need to know basis”. This particular phrase is also a mantra in object-oriented design as far as encapsulation is concerned, and it is why, in our case, the Credentials class is completely opaque – and why the Mechanism class is not. Each of these classes or, more generally, interfaces (as we are discussing the abstract classes in the Security namespace here) exposes nothing more than absolutely necessary in order to use them. When credentials have to be presented somewhere, they will be presented as either a pointer or a reference to an instance of this opaque class, which the code behind the scenes will know significantly more about than the code that lugs the instance around. Everything about these interfaces is on a need to know basis: you don’t need to know how to construct an instance of any of these classes, so you are not given the opportunity to even try. You don’t need to know what’s behind the credentials, so you can’t look. You don’t need to know how the mechanism works internally, so you don’t even get the slightest clue as to how it might work. All you get is what you absolutely need in order to use these classes.

Especially in distributed architectures (not what we are working on right now, but an interesting subject nontheless), this design mantra can be pushed to extremes. Sometimes, this is to the detriment of important functional aspects of the solution, such as its run-time performance: if in order to accomplish a task the program needs to take a “scenic route” to get the information it needs, something is evidently too opaque. The trade-off, however, may be in the maintainability of the program: taking short-cuts for the benefit of run-time performance may make the code very difficult to maintain in the long run. The point is: like everything, encapsulation should be used in moderation.

Opacity in Chausette

We start seeing how these two classes (Credentials and Mechanism), and the need to know, are different when we add the security context to the mix. The security context is the context in which authorization takes place. It is established between the server and the client application and it allows both the server and the client to mutually agree on what each may do with the other. The context is the result of a negotiation between the security mechanism implementations on either side of the link. In order to perform this negotiation, the two sides exchange tokes with each other – tokens we already know and have implemented a while ago. Neither SSPI nor GSS dictate how the tokens should be exchanged but both have similar concepts of contexts, so we’ll start by implementing the ones for SSPI and adapt our implementation to GSS afterwards.

First, let’s take a look at our new Context class:

#ifndef chausette_security_context_h
#define chausette_security_context_h
 
#include "Details/prologue.h"
 
namespace Vlinder { namespace Chausette { namespace Security { 
	class VLINDER_CHAUSETTE_SECURITY_API Context
	{
	public :
		virtual ~Context() = 0;
 
	protected :
		Context();
 
	private :
		// neither CopyConstructible nor Assignable
		Context(const Context &);
		Context & operator=(const Context &);
	};
}}}
 
#endif

As you can see, it is an almost exact copy of the Credentials class. It won’t stay that way but, for now, all we need to know is that such a thing as a context exists – just like all we needed to know about credentials is that they exist.

The negotiation that takes place between the client and the server in order to obtain a context has a variable number of steps and includes the exchange of tokens from either side. The mechanism can fail in several different ways to complete the creation of the context, and some of those failures may be fatal whereas others will not be fatal but will simply imply that more work needs to be done. We need to tell the calling code if anything went wrong, and what to do if that is the case.

The same is not true for credentials: if the mechanism can’t create the credentials requested, there can only be very few reasons why that might be so and, although we don’t handle those reasons at the moment, handling them will be significantly easier than handling the reasons for failing to create a context.

In one of my previous posts I explain what’s needed to handle an error correctly (in C, in this case): the calling code needs to know:

• what went wrong – which post-condition was not met, what action(s) did not take place, what action(s) failed
• why the function that was being called was called, and perhaps why it didn’t work
• how we got to where we were and how it went wrong

All of this information should be wrapped into an exception class, which will be the subject of my next installment. That exception, however, should only be thrown if there is an actual error: if the failure to create a complete context is simply a result of the negotiation process, the context may not be ready for use, but that is not an error. The context will therefore possibly need to be partial or, alternatively, the mechanism may need to keep it while it is being worked on.

Trade-offs

As far as opacity is concerned, however, the question becomes how much of this needs to be known to the user: does the user need to know that a given context is ready for use? Should the user have access to a context that is not ready for use yet? Should the user be allowed to interrupt a negotiation and start a new one? Should the user be allowed to have more than one on-going negotiation at the same time?

Each of these questions – or rather, the answers to them – have an impact on the opacity of the negotiation process, which is specific to the implementation of the mechanism; the opacity of the Context class and the opacity of the Mechanism class.

Let’s analyze each of these questions in more detail:

Does the user need to know that a given context is ready for use?

Of course, the user does need to know that a given context can be used. There will only be a doubt about this, however, if the user has access to contexts that cannot be used – e.g. partial contexts. Later during the life-time of the context a situation might occur in which the context is no longer usable – e.g. when it expires. The user should have a way to know this.

The easiest way to resolve this is to add an accessor to the base class, like this:

--- a/lib/security/Context.h
+++ b/lib/security/Context.h
@@ -9,6 +9,8 @@ namespace Vlinder { namespace Chausette { namespace Security {
 	public :
 		virtual ~Context() = 0;
 
+		virtual bool good() const throw() = 0;
+
 	protected :
 
 		Context();

Although that doesn’t tell the user why a context is not ok to be used, it does give her an indication that such is the case.

Should the user have access to a context that is not ready for use yet?

This is a trickier question: the negotiation process produces information in its intermediary states that needs to be stored somewhere and ultimately produces the context. The context itself has very little to do with those states, though many implementations will partially construct the context, doing so and returning the context may lead to confusion. On the other hand, it just might be the easiest way to handle the on-going negotiation process.

I see two alternatives: the information pertaining a particular negotiation can be stored in a new Negotiation object, which can be created using a startNegotiation method on the Mechanism class and would eventually be destroyed when the negotiation is either complete or abandoned; or the information can be stored in the Mechanism class itself, which would preclude the possibility of having more than one negotiation at a time, which leads to the next two questions.

Should the user be allowed to interrupt a negotiation and start a new one?

I think the answer to this should be a resounding “yes”. That means that regardless of the way we choose to implement negotiations, there must be a way to abort them. If we implement them using a Negotiation class, that will amount of destroying the class’ instance. If we keep the negotiation information in the Context class, destroying the context will destroy the information. If we keep it in the Mechanism class, we will have to add a method to abort the on-going negotiation.

Should the user be allowed to have more than one on-going negotiation at the same time?

I really don’t see much reason not to allow this, except if we choose to store the negotiation information inside the Mechanism object, in which case it would unnecessarily complicate our implementation.

These are the kinds of trade-offs that the “need to know basis” mantra takes us to: as we analyze our options, we find that the user will have to know that a negotiation is taking place, at least if we want him to be able to have any influence in the matter. As the architecture we have chosen since the beginning leaves the transport of tokens outside of the part of the solution we are currently working on (i.e. there is no concept of a “channel” that we can hand to the mechanism in order for it to talk to its counterpart on the other side) all such communication must necessarily go through the user’s code, which means we have to give her some control over the negotiation process. As long as that is the case, I believe that it should be done explicitly, so the negotiation itself should be encapsulated in a Negotiation object that, although completely opaque to the user, gives her a handle on the process so she can interrupt it, or manage it as she pleases.

Conclusion

Encapsulation in general, and opacity in particular, leads to trade-offs as you ask “what does the user need to know?” and “how should I tell her?”. In this particular case, we made the negotiation process transparent and thus left a large part of our implementation opaque, but we made the process transparent because the implementation is in the dark as far as communication is concerned. An alternative we haven’t discussed is to leave the negotiation opaque by bringing the communication into this part of the implementation. I’ll be happy to entertain a discussion on the merits of that solution if there’s any interest.

The code ends up looking like this:

diff --git a/lib/security/Details/Negotiation.cpp b/lib/security/Details/Negotiation.cpp
new file mode 100644
index 0000000..d3cda9e
--- /dev/null
+++ b/lib/security/Details/Negotiation.cpp
@@ -0,0 +1,9 @@
+#include "Negotiation.h"
+
+namespace Vlinder { namespace Chausette { namespace Security { namespace Details {
+	/*virtual */Negotiation::~Negotiation()/* = 0*/
+	{ /* no-op */ }
+
+	Negotiation::Negotiation()
+	{ /* no-op */ }
+}}}}
diff --git a/lib/security/Details/Negotiation.h b/lib/security/Details/Negotiation.h
new file mode 100644
index 0000000..ba78bfa
--- /dev/null
+++ b/lib/security/Details/Negotiation.h
@@ -0,0 +1,20 @@
+#ifndef chausette_security_details_negotiation_h
+#define chausette_security_details_negotiation_h
+
+namespace Vlinder { namespace Chausette { namespace Security { namespace Details {
+	class VLINDER_CHAUSETTE_SECURITY_API Negotiation
+	{
+	public :
+		virtual ~Negotiation() = 0;
+
+	protected :
+		Negotiation();
+
+	private :
+		// neither CopyConstructible nor Assignable
+		Negotiation(const Negotiation &);
+		Negotiation & operator=(const Negotiation &);
+	};
+}}}}
+
+#endif
diff --git a/lib/security/Mechanism.h b/lib/security/Mechanism.h
index 3e9fadd..95a2e2f 100644
--- a/lib/security/Mechanism.h
+++ b/lib/security/Mechanism.h
@@ -4,6 +4,7 @@
 #include "Details/prologue.h"
 #include <memory>
 #include <string>
+#include "Details/Negotiation.h"
 
 namespace Vlinder { namespace Chausette { namespace RFC1961 { class Token; }}}
 namespace Vlinder { namespace Chausette { namespace Security { 
@@ -14,8 +15,13 @@ namespace Vlinder { namespace Chausette { namespace Security {
 	public :
 		virtual ~Mechanism() = 0;
 
+		virtual std::auto_ptr< Details::Negotiation > startNegotiation() = 0;
 		virtual std::auto_ptr< Credentials > getCredentials(const std::string & principal, int flags) = 0;
-		virtual std::auto_ptr< Context > getContext(RFC1961::Token *& out_token, Credentials & credentials, const std::string & resource_name) = 0;
+		virtual std::auto_ptr< Context > getContext(
+			Details::Negotiation & negotiation,
+			RFC1961::Token *& out_token,
+			Credentials & credentials,
+			const std::string & resource_name) = 0;
 
 	protected :
 		Mechanism();
diff --git a/lib/sspi/Mechanism.cpp b/lib/sspi/Mechanism.cpp
index 0ffef6f..26df07a 100644
--- a/lib/sspi/Mechanism.cpp
+++ b/lib/sspi/Mechanism.cpp
@@ -40,6 +40,12 @@ namespace Vlinder { namespace Chausette { namespace SSPI {
 		delete data_;
 	}
 
+	/*virtual */std::auto_ptr< Security::Details::Negotiation > Mechanism::startNegotiation()/* = 0*/
+	{
+		//TODO implement this
+		return std::auto_ptr< Security::Details::Negotiation >();
+	}
+
 	/*virtual */std::auto_ptr< Security::Credentials > Mechanism::getCredentials(const std::string & principal, int flags)/* = 0*/
 	{
 		CredHandle cred_handle;
@@ -70,7 +76,11 @@ namespace Vlinder { namespace Chausette { namespace SSPI {
 		}
 	}
 
-	/*virtual */std::auto_ptr< Security::Context > Mechanism::getContext(RFC1961::Token *& out_token, Security::Credentials & credentials, const std::string & resource_name)/* = 0*/
+	/*virtual */std::auto_ptr< Security::Context > Mechanism::getContext(
+		Security::Details::Negotiation & negotiation,
+		RFC1961::Token *& out_token,
+		Security::Credentials & credentials,
+		const std::string & resource_name)/* = 0*/
 	{
 		Credentials & credentials_(dynamic_cast< Credentials& >(credentials)); // note: throws if the wrong type is given
 		CtxtHandle context_handle;
diff --git a/lib/sspi/Mechanism.h b/lib/sspi/Mechanism.h
index aa114b7..1c62b68 100644
--- a/lib/sspi/Mechanism.h
+++ b/lib/sspi/Mechanism.h
@@ -13,8 +13,13 @@ namespace Vlinder { namespace Chausette { namespace SSPI {
 		Mechanism(const std::string & package_name);
 		virtual ~Mechanism();
 
+		virtual std::auto_ptr< Security::Details::Negotiation > startNegotiation()/* = 0*/;
 		virtual std::auto_ptr< Security::Credentials > getCredentials(const std::string & principal, int flags)/* = 0*/;
-		virtual std::auto_ptr< Security::Context > getContext(RFC1961::Token *& out_token, Security::Credentials & credentials, const std::string & resource_name)/* = 0*/;
+		virtual std::auto_ptr< Security::Context > getContext(
+			Security::Details::Negotiation & negotiation,
+			RFC1961::Token *& out_token,
+			Security::Credentials & credentials,
+			const std::string & resource_name)/* = 0*/;
 
 	private :
 		// neither CopyConstructible nor Assignable

Much of this code still needs to be filled in but, before we do that, we have to figure out a way of handling errors – which we will do in the next installment. Until then, have fun!

                          

If, on the other hand, you want to do “core” programming, you have still different options in front of you.

The advantage you have in any of these cases, however, is that in tge majority of cases, the FLOSS tools have good reputations and are, for the most part, portable to other target platforms as well.

Toolchains

There are really only two FLOSS toolchains for Windows: Cygwin and Windows: MingW.

Cygwin

Cygwin is a Linux-like environment on Windows. It consists of a DLL (cygwin1.dll) and a large number of tools, all of which can be installed with a single installer, Cygwin Setup. This installer can be run any number of times, to add parts to the system and/or to update them. Cygwin is quite a bit more than just a toolchain (compiler, assembler, linker and related tools) but provides a complete environment to develop in, even including an X-windows environment. I recommend everyone who is familiar with the GNU system to install Cygwin on their Windows computer.

MinGW

MinGW is Minimalist GNU for Windows. It is a port of the entire toolchain (compiler, assembler, linker and related tools) to Windows and, unlike Cygwin, produces binaries that are native to Windows (binaries produced by the Cygwin toolchain need Cygwin to run). MinGW comes with MSYS, the Minimalist SYStem, which is a fork of Cygwin.

Most FLOSS tools that target Windows are based on the MinGW toolchain, and some ship it with their IDEs. Some don’t, in which case you can download and install it yourself using the MinGW installer, which you can download from SourceForge at the MinGW project page. Note, though, that the automated installed doesn’t necessarily install the latest versions of everything: manual installation instructions can be found here.

GUI Toolkits

There are several well-known FLOSS GUI toolkits available for Windows, most of which are either ports to Windows of well-known GNU/*NIX GUI toolkits, or toolkits that are free implementations of previously known Windows toolkits. I will treat only three in this article, each of which is very well-known and widely used: GTK+, wxWindows and Qt.

GTK+

GTK+ is a GUI framework written entirely in C. It is part of the GNU project and is at the heart of GNOME and the GIMP. It comes with a very interesting tool for GUI designed, called Glade, which generates an XML file that is interpreted at run-time to generate the on-screen GUI. Glade is licensed under GNU GPL, while GTK+ is licensed under GNU LGPL. GTK has bindings for many programming languages, including C++, and has a good reputation, being in-use for many well-reputed applications for several years now. It is under active maintenance by a team of volunteers. Windows ports exist and are also maintained.

Due to the fact that GTK+ is written in C, is part of the GNU project and uses only Free/Libre software, many of the basic building blocks that we tend to take for granted in a C++ environment that comes with the standard library, including the STL, have been re-implemented for GTK+. GTK+ is based on GLib, a low-level “core” library including ADTs and portability wrappers for things such as threads, and event loops; Pango, a text rendering library; Cairo, a 2D graphics rendering library; ATK, an accessibility toolkit. Each of these building blocks can be used separately (though there are dependencies between them).

For new projects written in C that need a GUI, this is probably the best choice for a GUI toolkit: there are many examples, the port to Windows is up-to-date and maintained and the port to most other platforms is up-to-date and maintained as well. The learning curve may be a bit steep to climb at first, but that is true for any full-fledged kit written in C. For new projects written in C++, there are GUI kits that are better suited for C++ development, which we will get to.

To obtain Windows binaries (and sources) of GTK+, you can go to this page for complete instructions.

wxWidgets

The wxWidgets project set out to create a low-cost, portable, open-source, easy-to-use GUI framework for C++ programming, and they have succeeded in doing so. It builds upon the native APIs for Windows and MacOS (though I’m not sure it uses Cocoa), and on GTK+ or Motif on *NIX.

wxWidgets consists of several libraries, including extended implementations of things you might find in the STL. It also contains libraries to handle XML, networking, text rendering, etc. although all this is quite a bit more encapsulated than it is in, for example, GTK+.

For new projects written in C++ that target different platforms, including (for example) smart phones, wxWidgets may be a good choice. I have no experience whatsoever with it, however, so I can neither recommend it, nor recommend not using it. I would suggest, though, that you take a look at wxWidgets compared to other toolkits including its last seciont: “Why You Shouldn’t Use wxWidgets”.

Go to the download page to download and install.

Qt

Now owned by Nokia, Qt was originally the closed-source library on which KDE was based. Qt has grown out of its closed-source roots and is now available under LGPL. It is a complete GUI toolkit written in C++ and ported across several platforms, including embedded ones. It even comes with its own windowing system for embdedded platforms.

Qt consists of its own build system, its own IDE, its own internationalization system, its own GUI design system, a large number of ready-made widgets and a complete set of libraries that do anything from networking to XML parsing, filesystem access, etc. This has, of course, its drawbacks as well as its advantages, but these “own implementations” are generally well-written and generic enough for general use.

Qt pioneered the signals/slots view of the Observer pattern, and uses its meta-object compiler to generate the code necessary to to maintain the signals and slots. Though with modern C++ compilers this is no longer necessary (see the Boost.Signal implementation of signals and slots for proof if you need it) at the time it was a judicious choice and now is only a minor nuisance.

For new C++ projects that need a GUI, this is the GUI toolkit I would personally use: it contains everything you need to quickly develop a GUI around your existing code and can sometimes come in handle to put an app together quickly. Qt’s implementation of the model-view-controller pattern is one of the best I’ve encountered so far, and allows you to completely de-couple the core implementation of your work from the GUI which, to me, is an absolute requirement for creating testable, maintainable software.

You can get the latest version at the Qt page.

Integrated Development Environments

There are several FLOSS IDEs on Windows. Obviously, I won’t count Microsoft Visual Studio Express Edition: it may be free as in beer, but it’s not libre or open source software. It wouldn’t work with either of the two toolchains I mentioned above anyway, btw, unless you have a nac for writing Makefiles and not getting support from your IDE to keep them up-to-date…

The ones I will look at are Code::Blocks, CodeLite, Anjuta, Qt Creator and Eclipse.

Code::Blocks and CodeLite

Right off the bat, I should say I’ve never used either of these two IDEs. They seem to be the IDEs of choice for wxWidgets, though, so I feel I should at least mention them. If you have anything, good or bad, to say about either of these IDEs, leave a comment.

Anjuta

I’ve used Anjuta for GTK development, and for development in C, on Linux (under GNOME) and on Windows. As far as I can tell, there are no recent ports to Windows of this IDE, however. The port that existed a while ago had some stability issues and should not be used in production environments. Under Linux, its behavior is a lot better.

I would not recommend using Anjuta in a FLOSS development environment on Windows. On Linux, as indicated, it’s another story – but that is not the subject of this article.

Qt Creator

While Qt Creator comes with Qt and, as I stated before, Qt is my GUI framework/toolkit of choice for new GUI applications in C++, Qt Creator is by no means my IDE of choice for such applications: although its integration with Qt is, obviously, very good, it assumes (usually correctly) that you’re doing Qt programming and is therefore less well-adapted to general-purpose programming. This may or may not be a problem, depending on whether you want to use the Qt build system and framework for non-GUI applications as well. As I usually don’t do that, Qt Creator is not my IDE of choice for non-GUI applications in a FLOSS development environment.

Eclipse

Eclipse makes no assumptions about anything (or hardly any about hardly anything) and is therefore well-suited for “core” programming as well as GUI programming. Due to its modular architecture, it is possible to extend Eclipse well beyond its roots as an IDE for Java. I have so far seen it used as an IDE for several embedded platforms, as well as for things that didn’t exactly look like an IDE.

For C and C++ programming, there’s an Eclipse plugin called CDT, which you can download from here. It does not come with a toolchain, but detects the presence of both Cygwin and MinGW, if they’re installed in any of the usual locations.

For GUI programming with Qt, there’s an Eclipse plugin that you can get from the Qt website. It looks like a similar plugin for GTK is dead in the water. For wxWidgets, there doesn’t seem to be a plugin, though there is a page on their wiki that explains how to use Eclipse (and suggests to use Code::Blocks in stead).

Personally, due to its completeness and the availability of many, many plugins, I’d go for Eclipse for an IDE in a FLOSS development environment.

Installation

I will consider the installation of two three toolchain/GUI toolkit/IDE combinations here: Cygwin with Eclipse, without a GUI; MinGW with Eclipse, without a GUI, MinGW with Eclipse and Qt, and QtCreator with Qt and MinGW. The installation procedure of each of these is described below.

Cygwin/Eclipse

In order to use Eclipse with Cygwin, you need the CDT plugin for Eclipse, and you need Cygwin to be installed. You can install Cygwin using Cygwin Setup. The installation process is pretty straight-forward. The toolchain is in the Devel category. Install at least g++ and its dependencies. That should include binutils and gcc. You’ll need gdb as well.

Once you have Cygwin installed, you might want to try building a simple “Hello, World!” program to check that everything works. Should not be a problem, but you might as well try.

Eclipse assumes Java (JRE) to be installed. Just go to java.sun.com and git the “Get Java” button if you don’t already have it (which is unlikely, but ye never know). It doesn’t need anything other than downloading and unzipping to run: there’s no installer.

Eclipse works with “work spaces” under which you have “projects” which contain your source code, as mapped on the drive. There’s a plugin for integration with Git (EGit) that you can download and install through Eclipse’s integrated installer.

Eclipse will detect the presence of Cygwin if it’s installed in the standard location.

MinGW/Eclipse

To install MinGW, you can either use the automated installer (hit the download button) or the manual installation procedure. The automated installer is easier to use but the manual installation procedure allows you to get more up-to-date tools. In either case, the tools in the toolchain are the same as for Cygwin.

The instructions for Eclipse are the same as well. You can install both toolchains on your computer and switch between the two if you like. Eclipse will detect them both of they are installed in the usual locations.

MinGW/Qt/Eclipse

Adding Qt to the MinGW/Eclipse mix is a question of installing Qt and installing the Qt plugin for Eclipse. This shouldn’t pose any problems – especially once you have Eclipse and MinGW installed.

MinGW/Qt/Qt Creator

Qt Creator comes with its own copy of MinGW, so you don’t need to install it separately. Just download and install the Qt SDK.

Conclusion

I hope this article has been helpful in selecting the tools for your FLOSS projects. If I’ve missed anything, or if you have any other questions, answers, comments or suggestions, please leave a comment.

                          

This article will explain why: I’ll first explain what a heap is, how it works, show you the complete implementation of a heap in C and analyze its computational complexity. I will also discuss the alternatives I had and why I chose to use a heap in stead.

What is a heap?

A heap (in our case) is a binary tree of which each node’s invariant is determined by a strict weak ordering predicate. I’ll explain what that means by explaining each of the terms I just used: invariant, predicate,strict weak ordering and binary tree.

What is an invariant?

An invariant, in its largest definition, is something that must always be the same. In our case, it is an assertion, or a set of assertions, that must always be true in order for our abstract data type (ADT, the heap) to be in a consistent state. For example: a trapezium is a shape that has four corners, each of which is connected to the next by a straight line, and those lines cannot cross. The invariants of a trapezium are:

• it has four corners
• each corner is connected to two other corners by a straight line
• none of the lines cross another line

.

The more invariants you add, the more you know about the object you’re adding invariants to. For example: a parallelogram is a trapezium of which each side is parallel to another side, which implies that each corner has the same angle as the opposing corner. Hence, the assertions

• each side is parallel to another side, and
• each corner has the same angle as the opposing corner

are both true, but are also mutually inclusive (if one is true, so is the other). Now, you can see that a parallelogram is really a special case of a trapezium – we’d model that in C++ as two classes, Trapezium and Parallelogram, which derive from each other (Parallelogram is derived from Trapezium). This implies that if you derive one class from another, the derived class must uphold all the invariants of the base class.

Let’s put this notion in pseudo-code:

class Trapezium
{
	void checkInvariants()
	{
		assert(this has four corners);
		assert(each corner is connected to two others by a straight line);
		assert(none of the lines cross any of the other lines);
		assert(at least one of the lines is parallel to another line);
	}
};

class Parallelogram : public Trapezium
{
	void checkInvariants()
	{
		Trapezium::checkInvariants();
		assert(each of the sides is parallel to another side);
		assert(each of the corners has the same angle as the opposing corner);
	}
};

We can push this notion a bit further by considering lozenges and rectangles: lozenges are parallelograms of which all the sides have the same length; rectangles are parallelograms of which all corners have a 90 degree angle:

class Lozenge : public Parallelogram
{
	void checkInvariants()
	{
		Parallelogram::checkInvariants();
		assert(all of the sides have the same length);
	}
};

class Rectangle : public Parallelogram
{
	void checkInvariants()
	{
		Parallelogram::checkInvariants();
		assert(all corners have a 90 degree angle);
	}
};

Now, you can see that by combining sets of invariants, we end up creating classes that don’t have any invariants of their own, but are still very useful:

class Square : Rectangle, Lozenge
{
	void checkInvariants()
	{
		Rectangle::checkInvariants();
		Lozenge::checkInvariants();
	}
}

In object-oriented design, invariants are very important, though sometimes they cannot be easily expressed in code. When they can be expressed in code, however, it is almost always useful to do so – though it is not always useful to check them during the execution of the code in production. When they are checked, there are a few caveats you should know about:

Invariants don’t only apply to the visible state of an object

Often, e.g. when implementing abstract data types such as heaps, the implementation can be wildly different from the visible state: the visible state of a heap, for example, is much the same as a stack, but sorted. Internally, unlike a stack, it is a tree which, unlike other trees, may be mapped into an array. In such a case, the invariants of an array apply, as do the invariants of our binary tree, but those of a “sorted stack” actually don’t apply, even though the ADT behaves as one.

Similarly, a circular buffer is usually implemented as an array and two pointers into the array and is often exposed as a contiguous space though an API. The buffer is mapped into the array using the two pointers, meaning the invariants of the buffer (such as the size of the writable space plus the size of the readable space is a constant) are different from the invariants of the implementation (such as both pointers always point within the array or one past the array).

Invariants may not hold at any moment, but should always hold after public operations have been performed

When implementing any non-trivial class that has invariants that can be expressed in code, those invariants invariably get broken while operations are in progress. Even externally visible invariants may be broken during such operations – and re-established afterwards. Take the circular buffer as an example (we’ll get to the heap when we start discussing its implementation): one of its invariants is that the total write space size plus the total read space size is a constant (i.e. if you have a circular buffer of N items, the number of items you can write when the buffer is empty is N, the number of items you can read when it is full is also N, and the if there are X items to read, you can still write N – X items). That constant is the capacity of the circular buffer. Now say you want to insert something into the buffer. Often, you’ll first “reserve a bucket” by advancing the write pointer. If at this moment you look at the write space, and there are X items already in the buffer, you get N – X – 1 because you can’t read from the bucket you’re writing to yet, but you can’t write to it from another write either. However, once you’re done with the bucket you were writing to, you release it and it becomes available for writing – thus re-establishing the invariant.

In practice, this means that when you’re implementing a class with an invariant, private members of the class cannot count on the invariant of the class being valid when they are called, while public and protected methods can. Also, private members don’t always re-establish the invariant when done, while protected and public methods do.

Similarly, when you’re working on an abstract data structure that uses locking internally to make it thread-safe, invariants should typically be re-established before a lock is released.

Now, you may remember about the three guarantees: the weak, strong and no-fail guarantee. These guarantees, formulated in terms of invariants are:

The weak guarantee:

the operation may fail, but will re-establish all invariants if it does so, such that the state of the object remains valid

the strong guarantee

the operation may fail, but in that case it has no effect (and therefore the object’s invariants still hold)

the no-fail guarantee

the operation cannot fail

You’ll remember that the weak guarantee is a minimum guarantee, so constructors must also obey it. I single out constructors because this means two-part construction (call a constructor, then call and initialization function) is generally a bad idea because the initialization function is typically used to establish the invariants for the first time.

What is a predicate?

A predicate is a function that returns a boolean. Usually, in C++, we use function objects (functors) for this:

struct Pred : unary_function< C, bool >
{
	bool operator()(C c) const
	{
		return /* something or other w.r.t. c */
	}
};

Sorted ADTs usually use binary predicates (predicates that take two arguments) for their sorting – i.e. to compare two values to each other. Our heap will do that too.

What is strict weak ordering?

See this explanation for an exact definition. Basically, it means that a predicate that provides this ordering provides the same kind of behavior as less-than does on numbers.

What is a binary tree?

A binary tree is a tree of which each node (except the leaf nodes) have (at most) two children. Trees like this are very common in computing because they are fairly easy to work with. Among other things, they can be stored in a contiguous array in an “Ahnentafel list”. This is a feature we will be using in our implementation as well. Another feature is that from any node, a direct path to the root is at most hops, where n is the number of nodes in the tree. This is called the depth of the tree. Due to these two advantages, the heap we are about to implement will have complexity for insertion and for removing an item from the heap, but, due to the invariants we will put in place, search will be in the worst case.

Putting it all together: the invariants of a heap’s binary tree

In the binary tree we’ll be implementing, each node will hold a value and that value will be smaller than the values of each if the node’s child nodes. I.e., we might model this as follows:

template < typename T >
class Node
{
public :
	Node(const T & t)
		: t_(t)
	{ /* no-op */ }
 
	void checkInvariants()
	{
		assert(!lhs_ || t_ < lhs_->t_);
		assert(!rhs_ || t_ < rhs_->t_);
	}
 
private :
	T t_;
	Node< T > * lhs_;
	Node< T > * rhs_;
};

Another invariant is that if a node has a right-hand-side node, it also has a left-hand-side node:

@@ -10,6 +10,7 @@
 	{
 		assert(!lhs_ || t_ < lhs_->t_);
 		assert(!rhs_ || t_ < rhs_->t_);
+		assert(!rhs_ || lhs_);
 	}
 
 private :

This latter invariant allows us to efficiently store the tree in an array – i.e. without losing any space. T, in the case of our first implementation, will be an integer.

Implementing the Heap

Like I mentioned above, the heap will be implemented as an array. We will need to know the number of elements in the array for almost any operation we might want to perform on the heap as well, so we will wrap the thing into a structure like this:

typedef struct Heap_struct
{
	int values_[HEAP_MAX_SIZE];
	unsigned int end_;
} Heap;

As with anything in C, this structure will need to be initialized and, possibly, finalized, so the first two functions of our API are straight-forward:

int Heap_init(Heap * heap)
{
	heap->end_ = 0;
 
	return HEAP_S_OK;
}
 
void Heap_fini(Heap * heap)
{ /* no-op */ }

Our API will consist of three other functions for now:

int Heap_push(Heap * heap, int value);
int Heap_pop(Heap * heap, int * value);
int Heap_find(Heap * heap, int value);

We’ll add other functions later.

Adding unit tests

As we’re writing an abstract data type, we really need to start testing it right away, and make sure our tests both cover as much as possible of our code, and don’t fail when we add new features to the code. We’ll be using Check for this, as we’ll be programming in C. Here’s what those tests look like for now:

#ifdef USING_CHECK
Suite * createTestSuite()
{
	Suite * s = suite_create("Heap");
 
	TCase * tc = tcase_create("Heap");
	suite_add_tcase(s, tc);
 
	return s;
}
 
int main()
{
	int fail_count;
 
	Suite * s = createTestSuite();
	SRunner * sr = srunner_create(s);
	srunner_run_all(sr, CK_NORMAL);
	fail_count = srunner_ntests_failed(sr);
	srunner_free(sr);
	return (fail_count == 0) ? EXIT_SUCCESS : EXIT_FAILURE;
}
#endif

Navigating the tree

Like I said before, we will be using an array to store our tree, which means that nodes are found as indices in the array. The root is at 0, and the left-hand-side child of any node N is found by (N * 2) + 1; the right hand side by (N * 2) + 2; the parent by (N - 1) / 2. Let’s take a look at the code that that gives:

#define Heap_nodeParent(h,n) (((n) - 1) / 2)
#define Heap_nodeLhs(h,n) (((n) * 2) + 1)
#define Heap_nodeRhs(h,n) (((n) * 2) + 2)
#define Heap_nodeSibling(h,n) (((n) % 2) ? ((n) + 1) : ((n) - 1))

There are two things you’ll perhaps note here: one is that these are macros, the second is that they each take a parameter they won’t actually use. The reason for using macros is that there’s really no reason to use functions for this – at least not in C. In C++, these would have been private methods of a class. The reason for the first parameter is that by convention, I pass the object being worked on as the first parameter to any function or anything that looks like a function – which would include macros. There’s no cost to this (except perhaps a bit of typing) and the benefit is that coding becomes more predictable. For the same reason, anything that looks like a function follows the same naming convention as a function, so my macros are not necessary all in uppercase.

So, right now, heap.h looks like this:

/* heap.h: a generic heap implementation in C
 * Copyright (C) 2010  Ronald Landheer-Cieslak
 *
 * This program is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <http://www.gnu.org/licenses/>. */
#ifndef vlinder_heap_h
#define vlinder_heap_h
 
#define HEAP_MAX_SIZE	64
 
#define HEAP_S_OK		0x00000000
#define HEAP_E_FULL		0x80000000
#define HEAP_E_EMPTY	0x80000001
 
#define HEAP_RESULT_IS_ERROR(s)	((s) & 0x80000000)
 
typedef struct Heap_struct
{
	int values_[HEAP_MAX_SIZE];
	unsigned int end_;
} Heap;
 
int Heap_init(Heap * heap);
void Heap_fini(Heap * heap);
 
int Heap_push(Heap * heap, int value);
int Heap_pop(Heap * heap, int * value);
int Heap_find(Heap * heap, int value);
 
#endif

and heap.c looks like this:

/* heap.c: a generic heap implementation in C
 * Copyright (C) 2010  Ronald Landheer-Cieslak
 *
 * This program is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <http://www.gnu.org/licenses/>. */
#include "heap.h"
#ifdef USING_CHECK
#include <check.h>
#endif
 
#define Heap_nodeParent(h,n) (((n) - 1) / 2)
#define Heap_nodeLhs(h,n) (((n) * 2) + 1)
#define Heap_nodeRhs(h,n) (((n) * 2) + 2)
#define Heap_nodeSibling(h,n) (((n) % 2) ? ((n) + 1) : ((n) - 1))
 
int Heap_init(Heap * heap)
{
	heap->end_ = 0;
 
	return HEAP_S_OK;
}
 
void Heap_fini(Heap * heap)
{ /* no-op */ }
 
#ifdef USING_CHECK
Suite * createTestSuite()
{
	Suite * s = suite_create("Heap");
 
	TCase * tc = tcase_create("Heap");
	suite_add_tcase(s, tc);
 
	return s;
}
 
int main()
{
	int fail_count;
 
	Suite * s = createTestSuite();
	SRunner * sr = srunner_create(s);
	srunner_run_all(sr, CK_NORMAL);
	fail_count = srunner_ntests_failed(sr);
	srunner_free(sr);
	return (fail_count == 0) ? EXIT_SUCCESS : EXIT_FAILURE;
}
#endif

As you can see, push, pop and find aren’t implemented yet, so we will start with find, which is perhaps the easiest to understand, though also the most complex, of the three.

As of here, code will be presented as patches whenever that makes sense.

Searching the heap

In order to search in the heap, we have to implement Heap_find. We’ll walk through the code bit by bit:

@@ -14,14 +14,30 @@
  * You should have received a copy of the GNU General Public License
  * along with this program.  If not, see <http://www.gnu.org/licenses/>. */
 #include "heap.h"
+#include <assert.h>
 #ifdef USING_CHECK
 #include <check.h>
+#include <stdlib.h>
 #endif
 
 #define Heap_nodeParent(h,n) (((n) - 1) / 2)
 #define Heap_nodeLhs(h,n) (((n) * 2) + 1)
 #define Heap_nodeRhs(h,n) (((n) * 2) + 2)
 #define Heap_nodeSibling(h,n) (((n) % 2) ? ((n) + 1) : ((n) - 1))
+#define Heap_nodeIsLeaf(h, n) ((h)->end_ <= Heap_nodeLhs(h, n))
+#define Heap_nodeIsRhs(h, n) (((n) % 2) == 0)
+#define Heap_nodeEq(h,n,v) ((h)->values_[n] == v)
+#define Heap_nodeLessV(h,n,v) ((h)->values_[n] < v)
+#define Heap_capacity(h) (sizeof((h)->values_) / sizeof((h)->values_[0]))

These macros allow us to query the heap and its nodes. As you can see, these are very dependent on the stored values being in an array of integers right now. We will revisit this later when the algorithmic parts are all set up.

The interesting part is this:

+
+static unsigned char Heap_depth(Heap * heap)
+{
+	unsigned int heap_size = Heap_capacity(heap);
+	unsigned char depth = 1;
+	while (heap_size /= 2) ++depth;
+
+	return depth;
+}
 
 int Heap_init(Heap * heap)
 {

This bit calculates the depth of the tree, which is where n is the number of nodes. We use this to check (in an assertion) that the stack we use to track our progress through the tree is large enough.

Now for the find function itself: I’ll put the whole code here first, then an annotated version.

@@ -33,12 +50,92 @@ int Heap_init(Heap * heap)
 void Heap_fini(Heap * heap)
 { /* no-op */ }
 
+int Heap_find(Heap * heap, int val)
+{
+	unsigned int root = 0, done = 0, retval = heap->end_;
+	int stack[HEAP_MAX_DEPTH], stackp = 0;
+	assert(Heap_depth(heap) <= (sizeof(stack) / sizeof(stack[0])));
+
+	if (root != heap->end_) while (!done)
+	{
+		if (Heap_nodeEq(heap, root, val))
+		{
+			done = 1;
+			retval = root;
+		}
+		else if (!Heap_nodeIsLeaf(heap, root) &&
+				  Heap_nodeEq(heap, Heap_nodeLhs(heap, root), val))
+		{
+			retval = Heap_nodeLhs(heap, root);
+			done = 1;
+		}
+		else if (!Heap_nodeIsLeaf(heap, root) &&
+				  Heap_nodeLessV(heap, Heap_nodeLhs(heap, root), val))
+		{
+			stack[stackp++] = root;
+			root = Heap_nodeLhs(heap, root);
+		}
+		else if (!Heap_nodeIsRhs(heap, root) &&
+				  Heap_nodeSibling(heap, root) < heap->end_)
+		{	// we're in a leaf node, visit the sibling, if one is there
+			root = Heap_nodeSibling(heap, root);
+		}
+		else if (stackp == 0)
+		{
+			done = 1;
+		}
+		else // go up the stack
+		{
+			do
+			{
+				root = stack[--stackp];
+			} while (root != 0 && (Heap_nodeRhs(heap, root) > heap->end_));
+			root = Heap_nodeRhs(heap, root);
+		}
+	}
+
+	return retval != heap->end_;
+}
+
 #ifdef USING_CHECK
+START_TEST(Test_Heap_find)
+{
+	int i, j;
+	Heap * heap = calloc(1, sizeof(Heap));
+	Heap_init(heap);
+	for (i = 0; i < HEAP_MAX_SIZE; ++i)
+	{	// a linear sorted set of values is a valid heap, which is good
+		// because that means we can test with that.
+		heap->values_[i] = i + 1;
+	}
+	// note we haven't changed end_ yet, so the heap is still empty
+	// k = 0;
+	for (i = 0; i < (HEAP_MAX_SIZE + 1); ++i)
+	{
+		heap->end_ = i;
+		for (j = 0; j < HEAP_MAX_SIZE; ++j)
+		{
+			if (j < i)
+			{
+				fail_unless(Heap_find(heap, j + 1));
+			}
+			else
+			{
+				fail_if(Heap_find(heap, j + 1));
+			}
+		}
+	}
+	Heap_fini(heap);
+	free(heap);
+}
+END_TEST
+
 Suite * createTestSuite()
 {
 	Suite * s = suite_create("Heap");
 
 	TCase * tc = tcase_create("Heap");
+	tcase_add_test(tc, Test_Heap_find);
 	suite_add_tcase(s, tc);
 
 	return s;

Of course, this chunk includes the unit test. The test basically creates a heap without using the insert function, which hasn’t been written yet, and loops over it to find all possible values.

Now let’s walk through the code. The function returns a boolean: non-zero if the value is found, 0 if it isn’t.

int Heap_find(Heap * heap, int val)
{
	unsigned int root = 0, done = 0, retval = heap->end_;
	int stack[HEAP_MAX_DEPTH], stackp = 0;
	assert(Heap_depth(heap) <= (sizeof(stack) / sizeof(stack[0])));

We use the stack to track the parent we were in. Though we could do that b just calculating the parent of the current node, it also allows us to easily see when we’re done, and it provides information when debugging.

Stacks aren’t all that interesting w.r.t. the algorithms: pushing and popping are both and they’re done in LIFO (last-in first-out) order. This particular implementation is just an array and a pointer (or rather: index) into it. Other algorithms are certainly possible, but the basic idea of a stack is that you can adapt almost any non-sorting linear container to become a stack, which is definitely useful.

	if (root != heap->end_)

Note this clause: we don’t start finding in an empty heap. If we would, the first thing we’d do would be to compare the value at the root with the one we’re looking for – which could result in unexpected behavior if the heap did not contain only integers.

while (!done)
	{
		if (Heap_nodeEq(heap, root, val))
		{

Right now, we check for equality. Later, we will check for equivalence in stead. If the value at the root of the current sub-tree is equal (or later: equivalent) to the one we’re looking for, we’re done.

			done = 1;
			retval = root;
		}
		else if (!Heap_nodeIsLeaf(heap, root) &&
			  Heap_nodeEq(heap, Heap_nodeLhs(heap, root), val))
		{

We take a bit of a short-cut here: if the left-hand-side child node is equal (later: equivalent) to what we’re looking for, we return it immediately. The reason why this is a short cut is that otherwise, we’d have to descend into the left-hand-side subtree if the left hand side is not greater than the value we’re looking for. It comes at the potential cost of an extra comparison, though. We could code an alternate version with one less comparison, if less-than comparison were expensive. It isn’t for integers, and shouldn’t be for any valid key.

			retval = Heap_nodeLhs(heap, root);
			done = 1;
		}
		else if (!Heap_nodeIsLeaf(heap, root) &&
			  Heap_nodeLessV(heap, Heap_nodeLhs(heap, root), val))
		{

When we get this far, we already know that the left-hand-side leaf is not equal, but it might be less than the value we’re looking for, in which case the value we are looking for might be in one of its children. In that case, we push the current root onto the stack and “dive” into the subtree.

			stack[stackp++] = root;
			root = Heap_nodeLhs(heap, root);
		}
		else if (!Heap_nodeIsRhs(heap, root) &&
			  Heap_nodeSibling(heap, root) < heap->end_)

From a left-hand-side leaf node we go to the right, to see if the value is there. Note that Heap_nodeSibling can move either way, so we need to know where we are before we call it, or we will get stuck here.

		{	// we're in a leaf node, visit the sibling, if one is there
			root = Heap_nodeSibling(heap, root);
		}
		else if (stackp == 0)
		{

If we’ve gotten this far and we don’t have a stack to climb up in, we’re at the root of the tree. That means we’re done.

			done = 1;
		}
		else // go up the stack
		{

Otherwise, we start going up the stack and going right.

			do
			{
				root = stack[--stackp];
			} while (root != 0 && (Heap_nodeRhs(heap, root) > heap->end_));
			root = Heap_nodeRhs(heap, root);
		}
	}
 
	return retval != heap->end_;
}

There’s something missing in this code – something our unit test didn’t pick up. We’ll find it shortly, but don’t use this code as-is just yet!

Now, the tree we’ve created in the test looks like this (in ASCII art):

                                                                   1
                                   2                                                               3
                   4                               5                               6                               7
           8               9              10              11              12              13              14              15
      16      17      18      19      20      21      22      23      24      25      26      27      28      29      30      31
    32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  60  61  62  63
64

As you can see, the tree is balanced, but not complete. It would have been the result of multiple calls to insert if insert were written and called with these numbers in sorted order.

Now, let’s say we use ,code>Heap_find to find number 35 in this heap. It would go over the following nodes, in this order: 1, 2, 4, 8, 16 32, 64, 33, 17, 34, 35. That's 11 steps - which is better than 35 steps which would have been a linear search. In fact, the works-cast scenario would be trying to find 63 in this heap, because it would actually be slightly worse than linear: in a linear search, it would have been found after 63 steps. In our case, it's found after 64 steps because it's the last leaf we look at - and 64 is one of the first leafs we look at.

You can also see that we've been a bit pessimistic when we allocated our stack: we will actually never have more than 5 roots on the stack (1, 2, 8, 16, 32) but we have seven spaces in the stack. There are two reasons for this: one is that the Heap_depth function is pessimistic: it returns the log2 of the power of two above the given value, so it will return one higher than necessary in our case ( but the function returns 7 because the power of two above 64 is 128). A simple patch can fix that:

@@ -34,7 +34,7 @@
 static unsigned char Heap_depth(Heap * heap)
 {
 	unsigned int heap_size = Heap_capacity(heap);
-	unsigned char depth = 1;
+	unsigned char depth = 0;
 	while (heap_size /= 2) ++depth;
 
 	return depth;

The second reason is that we never put the entire path on the stack: we only put the parent nodes on the stack when we've dived into the left-hand-side child of the node, so the current node is never on the stack. We don't want Heap_depth to reflect that, though, so if we want our stack to be less pessimistic, we have to fix the assertion:

@@ -54,7 +54,7 @@ int Heap_find(Heap * heap, int val)
 {
 	unsigned int root = 0, done = 0, retval = heap->end_;
 	int stack[HEAP_MAX_DEPTH], stackp = 0;
-	assert(Heap_depth(heap) <= (sizeof(stack) / sizeof(stack[0])));
+	assert(Heap_depth(heap) - 1 <= (sizeof(stack) / sizeof(stack[0])));

 	if (root != heap->end_) while (!done)
 	{

Fixing the bug

Like I said, there's a bug in the current version of the code that didn't get caught by our unit test - even though the coverage of the unit test couldn't be much better and it did perform 4160 iterations, which is a respectable amount of iterations for something like this. However, when describing the code (above), I left something out that may or may not have caught your attention. If it didn't, read the (annotated) code again before reading on.

The point of this little exercise is to show you that, though unit tests are good and will help you a lot when developing your own code, you should not forget to analyze your code thoroughly and think it through before putting it in production anywhere: you should put as much (relative) effort into your testing as you do in your development, if your testing is to be of the same quality as your development - otherwise, your development will turn out to be of the same quality as your testing.

Let's take a look at a test that will bring out the problem:

@@ -97,7 +97,7 @@ int Heap_find(Heap * heap, int val)
 }
 
 #ifdef USING_CHECK
-START_TEST(Test_Heap_find)
+START_TEST(Test_Heap_find_01)
 {
 	int i, j;
 	Heap * heap = calloc(1, sizeof(Heap));
@@ -129,12 +129,55 @@ START_TEST(Test_Heap_find)
 }
 END_TEST
 
+START_TEST(Test_Heap_find_02)
+{
+	int i, j;
+	static int heap_values[HEAP_MAX_SIZE] = {
+                                                                 1,
+                                 3,                                                              2,
+	         5,                              4,                              7,                              6,
+         9,              8,             11,             10,             13,             12,             15,             14,
+    17,     16,     19,     18,     21,     20,     23,     22,     25,     24,     27,     26,     29,     28,     31,     30,
+  33, 32, 35, 34, 37, 36, 39, 38, 41, 40, 43, 42, 45, 44, 47, 46, 49, 48, 51, 50, 53, 52, 55, 54, 57, 56, 59, 58, 61, 60, 63, 62,
+64,
+	};
+	Heap * heap = calloc(1, sizeof(Heap));
+	Heap_init(heap);
+	for (i = 0; i < HEAP_MAX_SIZE; ++i)
+	{	// a linear sorted set of values is a valid heap, which is good
+		// because that means we can test with that.
+		heap->values_[i] = heap_values[i];
+	}
+	// note we haven't changed end_ yet, so the heap is still empty
+	k = 0;
+	for (i = 0; i < (HEAP_MAX_SIZE + 1); ++i)
+	{
+		heap->end_ = i;
+		for (j = 0; j < HEAP_MAX_SIZE; ++j)
+		{
+			if (j < i)
+			{
+				int rv = Heap_find(heap, heap_values[j]);
+				fail_unless(rv);
+			}
+			else
+			{
+				fail_if(Heap_find(heap, heap_values[j]));
+			}
+		}
+	}
+	Heap_fini(heap);
+	free(heap);
+}
+END_TEST
+
 Suite * createTestSuite()
 {
 	Suite * s = suite_create("Heap");
 
 	TCase * tc = tcase_create("Heap");
-	tcase_add_test(tc, Test_Heap_find);
+	tcase_add_test(tc, Test_Heap_find_01);
+	tcase_add_test(tc, Test_Heap_find_02);
 	suite_add_tcase(s, tc);
 
 	return s;

As you can see, this test implements a heap that is designed to bring out the error: if you were to run the code with this test, the test would fail when trying to find the value 2 in the tree. The reason for that is that the find algorithm is missing a crucial step:

@@ -25,6 +25,7 @@
 #define Heap_nodeRhs(h,n) (((n) * 2) + 2)
 #define Heap_nodeSibling(h,n) (((n) % 2) ? ((n) + 1) : ((n) - 1))
 #define Heap_nodeIsLeaf(h, n) ((h)->end_ <= Heap_nodeLhs(h, n))
+#define Heap_nodeHasRhs(h, n) ((h)->end_ <= Heap_nodeRhs(h, n))
 #define Heap_nodeIsRhs(h, n) (((n) % 2) == 0)
 #define Heap_nodeEq(h,n,v) ((h)->values_[n] == v)
 #define Heap_nodeLessV(h,n,v) ((h)->values_[n] < v)
@@ -74,6 +76,17 @@ int Heap_find(Heap * heap, int val)
 			stack[stackp++] = root;
 			root = Heap_nodeLhs(heap, root);
 		}
+		else if (!Heap_nodeHasRhs(heap, root) &&
+				  Heap_nodeEq(heap, Heap_nodeRhs(heap, root), val))
+		{
+			retval = Heap_nodeRhs(heap, root);
+			done = 1;
+		}
+		else if (!Heap_nodeHasRhs(heap, root) &&
+				  Heap_nodeLessV(heap, Heap_nodeRhs(heap, root), val))
+		{
+			root = Heap_nodeRhs(heap, root);
+		}
 		else if (!Heap_nodeIsRhs(heap, root) &&
 				  Heap_nodeSibling(heap, root) < heap->end_)
 		{	// we're in a leaf node, visit the sibling, if one is there

This chunk of code checks the right-hand-side child node of the current node and handles the case where the value on the left-hand side of the node is greater than the one we're looking for, but the value on the right-hand side isn't.

If we take a close look at our invariants, we can see why this is important; the node invariant says that any given node's value should be smaller than the values of each of its children (if it has children). It doesn't say that the left-hand-side child should also be smaller than the right-hand-side child. The tree in our new test is a valid one, and inverts each of the values of adjacent children w.r.t. the tree in the first test. It brings this bug to light.

Pushing onto the heap

The algorithm to push a value onto the heap is pretty straight-forward: we use what's called a "sift-up" method in that we place the new value at a leaf in the tree and move it up (by swapping it with its parent) until the node invariant holds for the node it is in. An alternative method would have been to find a place to put it by "trickling down" from the root but because of the way we store our heap - in an array - we already know where the left-most leaf node is (it's at the end of the array: heap->values_[hea->end_]), which makes the sift-up method a lot easier to implement.

Let's take a look at the code:

int Heap_push(Heap * heap, int value)
{
	int retval = HEAP_S_OK;
	if (heap->end_ == Heap_capacity(heap))
	{
		retval = HEAP_E_FULL;
	}
	else
	{
		int curr = heap->end_++;
		heap->values_[curr] = value;
		while ((curr != 0))
		{
			if (Heap_nodeLess(heap, curr, Heap_nodeParent(heap, curr)))
			{
				Heap_nodeSwap(heap, curr, Heap_nodeParent(heap, curr));
				curr = Heap_nodeParent(heap, curr);
			}
			else
			{
				break;
			}
		}
	}
#ifndef NDEBUG
	Heap_checkInvariants(heap);
#endif
	return retval;
}<pre>Of course, the very first thing the function checks is whether it's actually possible to push anything on the heap - which it isn't if the heap is full. If it is full, it will return a valuye indicating that it is and do nothing more. Otherwise, it will put the new value at the end of the heap (which is the left-most available leaf node) and swap it up until it is either at the root, or its parent is smaller than it is.
 
Simple though this algorithm may be, it is useful to add unit tests for it. One test - a "smoke test" if you will - is to push random values onto the heap and make sure all of them can be found in the heap when done. It can be rather tedious to write such a smoke test, especially if you want more than one to be run (otherwise, how do you know that it's not just the random set of values you chose that happened to work?) but writing a script that generates the test case is easy enough: the following script generates 64 test cases, each with random values to put in the heap. <pre lang="sh">#! /bin/bash
rm -f heap.val.in heap.val.tcase.in
for ((i = 0; i < 64; ++i)); do
	echo "#include \"heap.val.${i}.in\"" >> heap.val.in
	echo "		tcase_add_test(tc, Test_Heap_push_rnd${i});" >> heap.val.tcase.in
	cat > heap.val.${i}.in <<EOF
START_TEST(Test_Heap_push_rnd${i})
{
	Heap * heap = calloc(1, sizeof(*heap));
	int i;
	int heap_values[HEAP_MAX_SIZE] = {
EOF
	dd bs=4 count=64 if=/dev/random of=tt
	od -A n -t x4 tt | sed -e 's/^ /		0x/' | sed -e 's/ /, 0x/g' | sed -e 's/$/,/g' >> heap.val.${i}.in
	cat >> heap.val.${i}.in <<EOF
	};
 
	Heap_init(heap);
	for (i = 0; i < HEAP_MAX_SIZE; ++i)
	{
		fail_unless(Heap_push(heap, heap_values[i]) == HEAP_S_OK);
	}
	// pushing anything in here now should fail
	fail_unless(Heap_push(heap, 0) == HEAP_E_FULL);
	for (i = 0; i < HEAP_MAX_SIZE; ++i)
	{
		fail_unless(Heap_find(heap, heap_values[i]));
	}
//	Heap_dump(heap, stdout);
	Heap_fini(heap);
	free(heap);
}
END_TEST
EOF
 
done

This is what one of those tests looks like:

START_TEST(Test_Heap_push_rnd54)
{
	Heap * heap = calloc(1, sizeof(*heap));
	int i;
	int heap_values[HEAP_MAX_SIZE] = {
		0x3196a3b8, 0xa53e4989, 0x23b36d80, 0xd324e8a7,
		0x73d71b10, 0xd5dea692, 0x2faf7ef1, 0x06515cb3,
		0xbd8affd4, 0x0eada2a5, 0xf068d6da, 0x5d720625,
		0x8077b60c, 0xdbd1fffd, 0x9f357c8c, 0x38ef64eb,
		0x4a1e1513, 0x891a7be9, 0xd738d651, 0xa715584c,
		0x223a4a78, 0xd02354a7, 0x503c4a3d, 0x5c5deb30,
		0xdff69472, 0xa5effd60, 0x2584ef20, 0x41f7f45b,
		0x846633f2, 0xf7bce84f, 0x70f1f224, 0xbc10b2ff,
		0xc1229e66, 0x67f83a40, 0xa7ef445c, 0xa25d9b97,
		0x2b86dea3, 0xb2b37e57, 0x450305ad, 0x3e194d93,
		0xe1422d07, 0xaa3c37f6, 0x92540227, 0x849b216a,
		0x18958218, 0xea89e7b1, 0x94b2a9e8, 0x7a244559,
		0x450879bd, 0x029059c9, 0xc121cb75, 0x73c21ced,
		0x09bba9f5, 0xdc1b66fc, 0xb8381cc8, 0xb7484b98,
		0xcc60b798, 0xc18653ef, 0x758ad5cc, 0x6c8e7d1d,
		0x54378939, 0x1562c20b, 0xe6c64b6d, 0xeb86a57b,
	};
 
	Heap_init(heap);
	for (i = 0; i < HEAP_MAX_SIZE; ++i)
	{
		fail_unless(Heap_push(heap, heap_values[i]) == HEAP_S_OK);
	}
	// pushing anything in here now should fail
	fail_unless(Heap_push(heap, 0) == HEAP_E_FULL);
	for (i = 0; i < HEAP_MAX_SIZE; ++i)
	{
		fail_unless(Heap_find(heap, heap_values[i]));
	}
//	Heap_dump(heap, stdout);
	Heap_fini(heap);
	free(heap);
}
END_TEST

These tests are generated by the Makefile if they don't exist.

Like I said, these are smoke tests. We also want to know whether the layout of the tree is as expected, which is why I wrote a function called Heap_dump to manually inspect the contents of a tree (I won't put the code of that here - it's a simple for loop). Also, at the end of each execution of the push function, the invariants of the nodes are checked to make sure the heap is still valid. The Heap_checkInvariants function looks like this:

static void Heap_checkInvariants(Heap * heap)
{
	int curr;
 
	for (curr = 0; curr < heap->end_; ++curr)
	{
		assert(Heap_nodeIsLeaf(heap, curr) || Heap_nodeLess(heap, curr, Heap_nodeLhs(heap, curr)));
		assert(!Heap_nodeHasRhs(heap, curr) || Heap_nodeLess(heap, curr, Heap_nodeRhs(heap, curr)));
	}
}

Of course, there is no need to follow the tree to check the invariants, as we already know where all the nodes are.

Popping from the heap

The algorithm we use for popping from the heap is a trickle-down algorithm: we swap the value at the root of the heap with the left-most leaf (which, again, is easy to find when you're storing the tree in an array as wel are) and trickle what was in the leaf down the tree until we find its final resting place. Here's the code:

int Heap_pop(Heap * heap, int * value)
{
	int root, child;
 
	if (heap->end_ == 0)
	{
		return HEAP_E_EMPTY;
	}
	else
	{ /* heap is not empty */ }
 
	Heap_nodeSwap(heap, 0, heap->end_ - 1);
	--heap->end_;
 
	root = 0;
	while (Heap_nodeLhs(heap, root) < heap->end_)
	{
		child = Heap_nodeLhs(heap, root);
		if ((Heap_nodeSibling(heap, child) < heap->end_) && 
			Heap_nodeLess(
				heap,
				Heap_nodeSibling(heap, child),
				child))
		{
			child = Heap_nodeSibling(heap, child);
		}
		else
		{ /* go left */ }
		if (Heap_nodeLess(heap, child, root))
		{
			Heap_nodeSwap(heap, child, root);
			root = child;
		}
		else
		{
			break;
		}
	}
 
	*value = heap->values_[heap->end_];
	return HEAP_S_OK;
}

Of course, this code comes with its own tests. Those tests are generated like the others. There's no real need for hand-written tests as we'd just be testing the same thing: regardless of the order in which items are put in the heap, we want them to come out in ascending order. The generating script now looks like this:

#! /bin/bash
rm -f heap.val.in heap.val.tcase.in
for ((i = 0; i < 64; ++i)); do
	echo "#include \"heap.val.${i}.in\"" >> heap.val.in
	echo "		tcase_add_test(tc, Test_Heap_push_rnd${i});" >> heap.val.tcase.in
	echo "		tcase_add_test(tc, Test_Heap_pop_rnd${i});" >> heap.val.tcase.in
	cat > heap.val.${i}.in <<EOF
#line 9 "mkvals.sh"
START_TEST(Test_Heap_push_rnd${i})
{
	Heap * heap = calloc(1, sizeof(*heap));
	int i;
	int heap_values[HEAP_MAX_SIZE] = {
EOF
	dd bs=4 count=64 if=/dev/random of=tt
	od -A n -t x4 tt | sed -e 's/^ /		0x/' | sed -e 's/ /, 0x/g' | sed -e 's/$/,/g' >> heap.val.${i}.in
	cat >> heap.val.${i}.in <<EOF
#line 19 "mkvals.sh"
	};
 
	Heap_init(heap);
	for (i = 0; i < HEAP_MAX_SIZE; ++i)
	{
		fail_unless(Heap_push(heap, heap_values[i]) == HEAP_S_OK);
	}
	// pushing anything in here now should fail
	fail_unless(Heap_push(heap, 0) == HEAP_E_FULL);
	for (i = 0; i < HEAP_MAX_SIZE; ++i)
	{
		fail_unless(Heap_find(heap, heap_values[i]));
	}
//	Heap_dump(heap, stdout);
	Heap_fini(heap);
	free(heap);
}
END_TEST
EOF
 
	cat >> heap.val.${i}.in <<EOF
#line 41 "mkvals.sh"
START_TEST(Test_Heap_pop_rnd${i})
{
	Heap * heap = calloc(1, sizeof(*heap));
	int i, prev, curr;
	int heap_values[HEAP_MAX_SIZE] = {
EOF
	dd bs=4 count=64 if=/dev/random of=tt
	od -A n -t x4 tt | sed -e 's/^ /		0x/' | sed -e 's/ /, 0x/g' | sed -e 's/$/,/g' >> heap.val.${i}.in
	cat >> heap.val.${i}.in <<EOF
#line 51 "mkvals.sh"
	};
 
	Heap_init(heap);
	for (i = 0; i < HEAP_MAX_SIZE; ++i)
	{
		fail_unless(Heap_push(heap, heap_values[i]) == HEAP_S_OK);
	}
	// pushing anything in here now should fail
	fail_unless(Heap_push(heap, 0) == HEAP_E_FULL);
	for (i = 0; i < HEAP_MAX_SIZE; ++i)
	{
		fail_unless(Heap_find(heap, heap_values[i]));
	}
	fail_unless(Heap_pop(heap, &prev) == HEAP_S_OK);
	for (i = 1; i < HEAP_MAX_SIZE; ++i)
	{
		fail_unless(Heap_pop(heap, &curr) == HEAP_S_OK);
		fail_unless(prev <= curr);
		prev = curr;
	}
	fail_unless(Heap_pop(heap, &curr) == HEAP_E_EMPTY);
//	Heap_dump(heap, stdout);
	Heap_fini(heap);
	free(heap);
}
END_TEST
EOF
 
done

Continuing from here: Predicates

With the tests and the heap in its current state, we have 132 tests (128 of which are automatically generated), all of which pass:

Running suite(s): Heap
100%: Checks: 132, Failures: 0, Errors: 0

This version is available here, but we're not quite done yet: we have that little thing called a predicate to handle.

Up to now, we've used three predicates without really naming them: the macros Heap_nodeEq, Heap_nodeLess and Heap_nodeLessV are all predicates. This really tells us only one thing, though: what our predicate should look like. I.e., it should be a function that takes two pointers to otherwise unknown objects (after all: this is C, we don't have templates to make the type known at compile-time) and return a boolean (nonzero for true, zero for false).

The changes to our code and test cases are really - perhaps surprisingly - limited:

@@ -29,12 +29,14 @@
 #define Heap_nodeHasLhs(h,n) !Heap_nodeIsLeaf(h,n)
 #define Heap_nodeHasRhs(h, n) (Heap_nodeRhs(h, n) < (h)->end_)
 #define Heap_nodeIsRhs(h, n) (((n) % 2) == 0)
-#define Heap_nodeEq(h,n,v) ((h)->values_[n] == v)
-#define Heap_nodeLess(h,n1,n2) ((h)->values_[n1] < (h)->values_[n2])
-#define Heap_nodeLessV(h,n,v) ((h)->values_[n] < v)
+#define Heap_nodeEq(h,n,v)													\
+	((!((h)->pred_((h)->values_[n], v))) && (!((h)->pred_(v, (h)->values_[n]))))
+#define Heap_nodeLess(h,n1,n2) 												\
+	((h)->pred_((h)->values_[n1], (h)->values_[n2]))
+#define Heap_nodeLessV(h,n,v) ((h)->pred_((h)->values_[n], v))
 #define Heap_nodeSwap(h,n1,n2)												\
 {																			\
-	int temp = (h)->values_[n1];											\
+	void * temp = (h)->values_[n1];											\
 	(h)->values_[n1] = (h)->values_[n2];									\
 	(h)->values_[n2] = temp;												\
 }
@@ -47,7 +49,7 @@ void Heap_dump(Heap * heap, FILE * out)
 	fprintf(out, "{ ");
 	for (i = 0; i < heap->end_; ++i)
 	{
-		fprintf(out, (i == (heap->end_ - 1) ? "%d " : "%d, "), heap->values_[i]);
+		fprintf(out, (i == (heap->end_ - 1) ? "%p " : "%p, "), heap->values_[i]);
 	}
 	fprintf(out, "}\n");
 }
@@ -59,8 +61,12 @@ static void Heap_checkInvariants(Heap *
 
 	for (curr = 0; curr < heap->end_; ++curr)
 	{
-		assert(Heap_nodeIsLeaf(heap, curr) || Heap_nodeLess(heap, curr, Heap_nodeLhs(heap, curr)));
-		assert(!Heap_nodeHasRhs(heap, curr) || Heap_nodeLess(heap, curr, Heap_nodeRhs(heap, curr)));
+		assert(
+			Heap_nodeIsLeaf(heap, curr) ||
+			Heap_nodeLess(heap, curr, Heap_nodeLhs(heap, curr)));
+		assert(
+			!Heap_nodeHasRhs(heap, curr) ||
+			Heap_nodeLess(heap, curr, Heap_nodeRhs(heap, curr)));
 	}
 }
 
@@ -74,9 +80,10 @@ static unsigned char Heap_depth(Heap * h
 }
 #endif
 
-int Heap_init(Heap * heap)
+int Heap_init(Heap * heap, Heap_pred pred)
 {
 	heap->end_ = 0;
+	heap->pred_ = pred;
 
 	return HEAP_S_OK;
 }
@@ -84,7 +91,7 @@ int Heap_init(Heap * heap)
 void Heap_fini(Heap * heap)
 { /* no-op */ }
 
-int Heap_push(Heap * heap, int value)
+int Heap_push(Heap * heap, void * value)
 {
 	int retval = HEAP_S_OK;
 	if (heap->end_ == Heap_capacity(heap))
@@ -114,7 +121,7 @@ int Heap_push(Heap * heap, int value)
 	return retval;
 }
 
-int Heap_pop(Heap * heap, int * value)
+int Heap_pop(Heap * heap, void ** value)
 {
 	int root, child;
 
@@ -157,7 +164,7 @@ int Heap_pop(Heap * heap, int * value)
 	return HEAP_S_OK;
 }
 
-int Heap_find(Heap * heap, int val)
+int Heap_find(Heap * heap, void * val)
 {
 	unsigned int root = 0, done = 0, retval = heap->end_;
 	int stack[HEAP_MAX_DEPTH], stackp = 0;
@@ -216,15 +223,19 @@ int Heap_find(Heap * heap, int val)
 }
 
 #ifdef USING_CHECK
+static int Test_less(void * lhs, void * rhs)
+{
+	return (lhs < rhs);
+}
 START_TEST(Test_Heap_find_01)
 {
 	int i, j;
 	Heap * heap = calloc(1, sizeof(Heap));
-	Heap_init(heap);
+	Heap_init(heap, Test_less);
 	for (i = 0; i < HEAP_MAX_SIZE; ++i)
 	{	// a linear sorted set of values is a valid heap, which is good
 		// because that means we can test with that.
-		heap->values_[i] = i + 1;
+		heap->values_[i] = (void*)(i + 1);
 	}
 	// note we haven't changed end_ yet, so the heap is still empty
 	for (i = 0; i < (HEAP_MAX_SIZE + 1); ++i)
@@ -234,11 +245,11 @@ START_TEST(Test_Heap_find_01)
 		{
 			if (j < i)
 			{
-				fail_unless(Heap_find(heap, j + 1));
+				fail_unless(Heap_find(heap, (void*)(j + 1)));
 			}
 			else
 			{
-				fail_if(Heap_find(heap, j + 1));
+				fail_if(Heap_find(heap, (void*)(j + 1)));
 			}
 		}
 	}
@@ -260,11 +271,11 @@ START_TEST(Test_Heap_find_02)
 64,
 	};
 	Heap * heap = calloc(1, sizeof(Heap));
-	Heap_init(heap);
+	Heap_init(heap, Test_less);
 	for (i = 0; i < HEAP_MAX_SIZE; ++i)
 	{	// a linear sorted set of values is a valid heap, which is good
 		// because that means we can test with that.
-		heap->values_[i] = heap_values[i];
+		heap->values_[i] = (void*)heap_values[i];
 	}
 	// note we haven't changed end_ yet, so the heap is still empty
 	for (i = 0; i < (HEAP_MAX_SIZE + 1); ++i)
@@ -274,12 +285,12 @@ START_TEST(Test_Heap_find_02)
 		{
 			if (j < i)
 			{
-				int rv = Heap_find(heap, heap_values[j]);
+				int rv = Heap_find(heap, (void*)heap_values[j]);
 				fail_unless(rv);
 			}
 			else
 			{
-				fail_if(Heap_find(heap, heap_values[j]));
+				fail_if(Heap_find(heap, (void*)heap_values[j]));
 			}
 		}
 	}
@@ -294,16 +305,16 @@ START_TEST(Test_Heap_push_01)
 	Heap * heap = calloc(1, sizeof(*heap));
 	int i;
 
-	Heap_init(heap);
+	Heap_init(heap, Test_less);
 	for (i = 0; i < HEAP_MAX_SIZE; ++i)
 	{
-		fail_unless(Heap_push(heap, i + 1) == HEAP_S_OK);
+		fail_unless(Heap_push(heap, (void*)(i + 1)) == HEAP_S_OK);
 	}
 	// pushing anything in here now should fail
 	fail_unless(Heap_push(heap, 0) == HEAP_E_FULL);
 	for (i = 0; i < HEAP_MAX_SIZE; ++i)
 	{
-		fail_unless(heap->values_[i] == i + 1);
+		fail_unless(heap->values_[i] == (void*)(i + 1));
 	}
 	Heap_fini(heap);
 	free(heap);
@@ -325,16 +336,16 @@ START_TEST(Test_Heap_push_02)
 64
 	};
 
-	Heap_init(heap);
+	Heap_init(heap, Test_less);
 	for (i = 0; i < HEAP_MAX_SIZE; ++i)
 	{
-		fail_unless(Heap_push(heap, HEAP_MAX_SIZE - i) == HEAP_S_OK);
+		fail_unless(Heap_push(heap, (void*)(HEAP_MAX_SIZE - i)) == HEAP_S_OK);
 	}
 	// pushing anything in here now should fail
 	fail_unless(Heap_push(heap, 0) == HEAP_E_FULL);
 	for (i = 0; i < HEAP_MAX_SIZE; ++i)
 	{
-		fail_unless(heap->values_[i] == heap_values[i]);
+		fail_unless(heap->values_[i] == (void*)heap_values[i]);
 	}
 	Heap_fini(heap);
 	free(heap);

Our predicate-wielding macros have become a bit more complicated because they call functions now, and the "equals" macro has become an "equivalent" macro (predicate fails on the two values in both directions), but that's really all there is to it.

Final details: duplicate values

There is one thing we haven't explicitly tested yet, but we should: we haven't tried putting two or more equivalent values on the heap. Our random tests could conceivably run into the case from time to time, but if the system's random number generator (in /dev/random) is any good, the likelihood of that happening is pretty slim (one in about sixty-eight million). So, what happens if we do this:

START_TEST(Test_heapPush_duplicates)
{
	Heap * heap = calloc(1, sizeof(*heap));
	int i;
 
	Heap_init(heap, Test_less);
	for (i = 0; i < HEAP_MAX_SIZE; ++i)
	{
		fail_unless(Heap_push(heap, 0) == HEAP_S_OK);
	}
	// pushing anything in here now should fail
	fail_unless(Heap_push(heap, 0) == HEAP_E_FULL);
	for (i = 0; i < HEAP_MAX_SIZE; ++i)
	{
		fail_unless(heap->values_[i] == 0);
	}
	for (i = 0; i < HEAP_MAX_SIZE; ++i)
	{
		void * val;
		fail_unless(Heap_pop(heap, &val) == HEAP_S_OK);
		fail_unless(val == 0);
	}
	fail_unless(Heap_pop(heap, 0) == HEAP_E_EMPTY);
	Heap_fini(heap);
	free(heap);
}
END_TEST

The answer is this:

Running suite(s): Heap
assertion "Heap_nodeIsLeaf(heap, curr) || Heap_nodeLess(heap, curr, Heap_nodeLhs(heap, curr))" failed: file "heap.c", line 66, function: Heap_checkInvariants
99%: Checks: 133, Failures: 0, Errors: 1
heap.c:365:E:Heap:Test_heapPush_duplicates:0: (after this point) Received signal 6 (Aborted)

meaning one of the invariants failed.

There are two ways to handle this: either we change the invariant to allow duplicate values, in which case the invariant becomes "the parent is smaller than or equivalent to each of its children", or we detect the duplicate value when it's inserted into the heap and fail at that point, with an error. In order to decide which one we'll do, we'll consider three things: the requirements our heap has to meet, the risk of not meeting those requirements in either implementation, and the consequences of both implementations if executed correctly.

In terms of requirements, the basic requirements for our heap are pretty straight-forward: we want a push onto the heap, and a pop from the heap, to have complexity, and a search in the heap to have complexity on average, and complexity in the worst case. In our current implementation, pops are slightly more expensive that pushes because pushes only perform one comparison per node encountered, and comparisons in the worst case. The pop performs two comparisons per node (one with the right-hand-side child, one with the left-hand-side child) and comparisons in the worst case. The find algorithm does three comparisons in the worst case, but may go through the whole tree before it gives up, so it's in the worst case. This is still within our complexity requirements, but we should avoid making it any more expensive if we can.

In terms of guarantees, we need both push and pop to give the strong guarantee: neither should break the invariants and neither should have any effect on the visible state of the heap if they fail: a pop should always either remove a single object from the heap - and that object should be the smallest object on the heap - or fail if the heap is empty; and a push should always either put a single object on the heap or fail if the heap is full. Find should never fail, nor change the state of the heap (nor break any invariants).

Let's consider the way the push algorithm works: it puts the new value in the left-most vacant leaf and propagates it up the tree toward to root. In doing so, it only considers the nodes on the path from the left-most vacant leaf to the root, disregarding any other nodes. This is why the complexity of push is . The most naive implementation of push that would attempt to detect duplicate values would simply detect a duplicate value on that path an report an error if one is found, rolling back the changes it made up to that point. Such an implementation would retain its complexity - but would it reliably detect duplicates? Let's consider the following bit of code (and imagine that we're using ints in stead of void*s again):

Heap_push(heap, 0);
Heap_push(heap, 1);
Heap_push(heap, 1);

This would construct a tree with three nodes: zero at the root, one in each of its children. More importantly, this would not fail because the second time we insert 1, the left-most vacant leaf is the right-hand-side leaf of the root node, so the algorithm would only compare with the root. If we'd want it to compare with the other node containing 1, we'd have to compare with the new node's sibling as well. This is all fine and dandy, until the tree becomes a bit more complex, e.g. by inserting the following values in an empty heap:

Heap_push(heap, 0);
Heap_push(heap, 10);
Heap_push(heap, 12);
Heap_push(heap, 20);
Heap_push(heap, 15);
Heap_push(heap, 17);
Heap_push(heap, 15);

This results in the following tree:

       0
  10      12
20  15  17  15

Note that this tree is still perfectly valid according to the current invariants, but would cease to be so at the first pop, which would result in the following tree:

      10
  15      12
20  15  17

According to our current invariants, this tree is not valid, because the upper node with value 15 has a child that is not greater than itself (i.e. it is not less than both of its children). That means that, in this case, pop would break the invariant - not push.

The only way to reliably avoid this would be to search the heap before inserting the new value but, as we've seen, searching the heap is costlier than inserting a value into it: this would make insertion . Another option that would work for integers is to detect duplicate values at the moment they are popped, by keeping the latest popped value and comparing with it before giving the value back. That won't work for real objects, though, for two reasons: first of all, our implementation doesn't own the objects in the heap but only has pointers to them. This could be handled by having the heap own its contents, but that would bring all sorts of problems with it, which I won't go into right now. The second reason is that for real objects we don't check equality, but equivalence. As the heap does not provide FIFO order for equivalent values (even in our current implementation) there is really no way to predict the order in which two equivalent values would come out of the heap, so which pop would be quenched by this approach would also be unpredictable - and we would still have to alter the invariants to allow for such an approach.

Suffice it to say that remaining within our performance requirements, this feature cannot be implemented reliably.

Simply not implementing the feature and leaving the state as-is is not really an option: any self-respecting implementation of an abstract data type should not delegate the responsibility of its own integrity - the validity of its internal invariants - to its users. Not changing the current state of affairs and keeping the invariants as they are would allow the client code to break the invariants simply by inserting the same value into the heap twice. Forcing the client code to detect that before they push a value onto the heap would mean forcing them to perform a find before each push, effectively creating a situation in which pushing is both error-prone and .

So, our only remaining option is to change the invariant of the heap, allowing duplicate values and, therefore, allowing equivalence as well as the less-than relationship between parent and child. This patch will do that:

@@ -63,10 +63,14 @@ static void Heap_checkInvariants(Heap *
 	{
 		assert(
 			Heap_nodeIsLeaf(heap, curr) ||
-			Heap_nodeLess(heap, curr, Heap_nodeLhs(heap, curr)));
+			Heap_nodeLess(heap, curr, Heap_nodeLhs(heap, curr)) ||
+			(!Heap_nodeLess(heap, curr, Heap_nodeLhs(heap, curr)) &&
+			 !Heap_nodeLess(heap, Heap_nodeLhs(heap, curr), curr)));
 		assert(
 			!Heap_nodeHasRhs(heap, curr) ||
-			Heap_nodeLess(heap, curr, Heap_nodeRhs(heap, curr)));
+			Heap_nodeLess(heap, curr, Heap_nodeRhs(heap, curr)) ||
+			(!Heap_nodeLess(heap, curr, Heap_nodeRhs(heap, curr)) &&
+			 !Heap_nodeLess(heap, Heap_nodeRhs(heap, curr), curr)));
 	}
 }
 

Now, if the client code does not want to allow duplicate values, it will have to perform a find on the heap before inserting but if it does want to allow duplicate values, it can simply insert them. If wants to go for the cheaper option of having duplicate values filtered at the pop-end of the heap, it can do that with full knowledge of what equivalence means (as the client code provides the predicate) and of how element life-time is handled (as that is also handled by the client code). Our heap makes no assumptions on either.

Conclusion

We've created a monster!

Actually, no. We've created a fairly efficient implementation of a heap with all the bells and whistles, that also happens to be fairly generic. It's written in C, but the same thing would certainly have been possible in C++ (and I might be tempted to write a version in C++ at some point). We've seen invariants in action, analyzed the computational complexity of our algorithm and solved a few problems along the way.

The generic version of the heap implementation is available as version 1.1.00, here.

                          

In fact, come to think of it, most of the technologies I work with on a daily basis have a woman’s influence in its history. Women have traditionally played an important role in anything ranging from the esthetic to the very, very technical. One of the women in the team I referred to earlier was in charge of quality assurance for the team while the other worked on a firmware, written in C.

So why is it that computing tends to be such a masculin environment (in that there are a lot of men working in this field)? Maybe it’s just because women are smarter than men..? They’ve moved on to greener pastures where their superior intellect is more needed, leaving us to do the footwork. The few women remaining in the field are just there to keep an eye on us…

Nah.., that can’t be it, can it..?

                          

Say the chance of any given bit flipping is , the number of bits in a byte is and the number of bytes in a message is . This means that the chance of a bit not flipping is and the chance of getting a message through without any flipped bits is , which, depending on the values of and can either be very high, or very low, so let’s fill in a few values to get an idea of what this means. Let’s say that the chances of a bit flipping are about one in a million, we have eight bits per byte and our messages are 256 bytes in length. The message may be a bit larger than what you’d usually find on a serial connection, but we’re going for a case in which there’s an actual payload to send, so I think it’s reasonable. With these numbers, our little formula, works out to a value for of 0.997954. That is: with a one-in-a-million chance of any bit flipping, one message out of ever 500 is corrupt. If that one message was “shields down” and, due to the bit flip, becomes “eject the warp core”, and the Enterprise is dead in space.

So, the idea of “tell me twice” is to reduce the chances of such errors happening by reducing the chance that a bit flip would have any effect. This is based on the idea that the chances of the same bit flipping in two consecutive chances are far lower than the one-in-five-hundred we just came up with, even it the chances of a bit flipping are still the same. To understand this, we need to know what the chances are that a specific bit in a message is flipped. That chance is , which is smaller that the chance of any individual bit flipping (because all the other bits have to not flip). In our case, this works out to which is about one in 1,002,049.

Now, the probability that a message is followed by another message that has the same bit flipped is the product of the probability that a message has a flipped bit (one in 500) and the probability that a message has a given bit flipped (one in a little over a million): , which in our case works out to which is about one in a half billion.

Of course, added to this is the fact that most serial protocols also have some kind of CRC checking, there’s usually a parity bit in there somewhere, etc. All these individually rather weak methods, together, make it nearly impossible for a corrupt message to come through undetected – because for any broken message to come through undetected, all of the error detection features have to fail, so as long as they don’t all check the same thing the same way, the chances of that happening is the product of the chances of each individual method failing (i.e. checking the parity five times doesn’t make your chances of finding an undetected error any better, but checking the parity and four other things does).

                          

In the scenario he described, there’s really little chance of creating quality code, unless…

I avoid contracts like the one John described like the plague: if the client doesn’t know what they want, there’s little chance for me to come up with a satisfactory solution. Then again, I am in a slightly different business than John is: I don’t make applications for end-user devices. In stead, I make the software that runs under or behind it – embedded software, firmware and server software (either I work on the very small, or I work on the very big). In the last decade, I’ve written code for servers and code for embedded devices, but of that code, almost nothing has been visible to the end-user. Some has been visible to an operator on a packaging line, but that is just a small minority of the code I’ve written. However, I’ve worked with many of the same technologies as John apparently has: web services to get data, databases, etc. are among the things I’ve worked with, as he has. So, how do I go about avoiding contracts as the one John described?

The one thing in short-term contracts is to avoid surprises: you have to work with a third part, you have to know it from the outset. Nothing should be assumed: in any application, data has to come from somewhere (a source) and go somewhere (a sink); events need to be triggered and those triggers also need to come from somewhere; relations between all datasets need to be established from the outset, etc. Like I said: I don’t usually work on end-user applications, but from my point of view, there are a few lessons to be learned from John’s experience:
questions that needed to be asked, weren’t. The feeling the execs wanted to have come across that the app was going to be “simple” was accepted at face value. I can understand that John didn’t want to seem uncomfortable with that assertion, but seeing as he was the specialist in the room, it was clearly up to him to decide whether the app was simple or not – and it was up to him to show that such was not the case.

“But it’s OK,” Gorilla Mart Executive #1 says. “The app is simple.”

This, of course, is the cue for “I’ll be the judge of that!” or something in that order. The point is that there can be only one judge of the simplicity of an app, and that judge is the specialist. I’m always happy to explain how I intend to solve a given problem, and I am always happy to accept suggestions as to the solution. To go from there to accepting a solution from an exec, who may well have an MBA, is a step I am not willing to take without good arguments.

Now, I don’t want to come over as a know-it-all, nor do I want to judge the way other people handle their contracts, so I’ll propose a few anecdotes of my own.

Anecdote #1: Purdy Please?!

This was about a year ago. I’d put a notice up on this site that I was available for a new contract and got a phone call a few days later, if I would accept a two-week contract to interface an existing system to a brand-new web service for it to obtain geo-location data. The contract was to be done post-haste, so the person searching for a developer was in a hurry.

I have a list of developers that I can call if I have a contract that is too big for me to handle by myself, or if I don’t have time to do the contract myself. These are all people I’ve worked with before and for whom I can vouch in terms of their being good programmers. When I had put the notice up on my site that I was available, I didn’t expect to get calls quite that quickly, so with the deadline as close as it was, there was no way I could accept the contract myself. When pushed a bit (the guy was 300 km away from his client and needed to get a developer right now) I took out my list and made a few calls. I found a suitable developer, and earned a commission for the referral.

Crisis averted, and everybody happy: the programmer that got to do the job, the HR person looking for a programmer, the client and me all got what we wanted, and three out of four were a bit richer after the deal – the fourth had his product, so will be a bit richer by now as well.

Anecdote #2: What do you mean: “what do you need?”?

A few months ago, I got a call asking me to develop the firmware for a new product. The call lasted about thirty minutes and ended with me asking for some material to at least give me a feel of what was needed – something I could base a draft architecture on.

They didn’t have any such documentation – actually wanted me to come up with some. Naturally, I can’t do that: I don’t know the market for the device, nor do I know enough about marketing to come up with that kind of material. All I could do is give them a few pointers, and tell them to come back to me when they had some more information.

Now, this is a case in which no money exchanged hands and the amount of work on my part was really minimal: they basically got an expert opinion on a few questions they had and were happy with the answers they got. They also knew they had a bit more homework to do. Me accepting the contract without that homework being done first would be a waste of my time and theirs – and as my time is their money, they understood that I was saving them money with my questions.

When their homework is done and they do come back to me, the time I spent will have been a good investment on my part. If they don’t do their home work or they don’t come back to me, it will still have been a good investment on my part, but it will not have had the hoped-for return.

Anecdote #3: Questions, questions…

There are few clients that like your work enough to pay you in advance for time they might not even use – just so they can call you any time during the day and ask a few questions. One such client wanted me to write a specification from a set of PowerPoint slides. Happy to oblige, I went through the PowerPoint slides and came up with a long list of questions. My continuing the work on the specification depends on getting the answers to those questions.

Now, these questions aren’t technical in nature, but PowerPoint slides have a tendency to be vague and technical specifications can’t really have that luxury, so choices have to be made at some point. E.g., when defining a security profile of a new device, do you want to emphasize confidentiality, integrity, authentication or non-repudiation? In embedded devices, you usually don’t have enough space to have them all and I can’t make such decisions on behalf of my clients – nor can I describe all options and have the development team make the choices (although I can describe and explain alternatives).

Now, does this mean that every time a client comes up with a vague question, I tell them to do their homework and go on my merry way? No. I am not that financially independent. It does mean, though, that creating software-intensive systems requires real input from the system’s stakeholders. I’m happy to work with stakeholders and think different scenarios through with them, and I’m very happy to explain alternatives to them, to analyse possible repercussions of different choices, etc. but I cannot, and will not, take their decisions for them.

Of course, all three of these anecdotes didn’t actually result in me investing any significant amounts of time (except for the third one, in which I spent the better part of three days analysing the presentation and the repercussions for the existing architecture) and therefore didn’t come at great cost to the client. In each case, the customer was helped along the way to the end-result they were going for, and in the first case, the project was eventually executed while the other two are still pending.

So, how is this approach different from the one described by John? Well, aside from none of these cases being particularly frustrating for me or for the client, I don’t get “pumped” – I hardly ever get excited – for a job: getting too excited about anything usually ends up in disappointments for everyone involved. Now, that doesn’t mean I haven’t made my share of mistakes: committing to deadlines I couldn’t live up to, working nights, etc. to “get the job done” and producing code that would have smelled the way it looked if code had any smell to it at all, but – knock on wood – those days are (hopefully) over… almost.

                          

In this installment, we will continue our implementation of GSSAPI/SSPI, this time on Windows, where we’ll try to get some credentials.

We will look at two topics this time: first, we’ll look at data encapsulation, after which we’ll look at when RAII is a bit too much, and how to handle that.

Data encapsulation

One of the main reasons we’re going to the trouble of creating an abstract factory is because we want the users of our code to be pretty close to impervious to the way security is implemented. Most notably, we don’t want the user to be obliged to know anything about SSPI or GSSAPI and we don’t want them to have to include any of the associated headers – or even necessarily have those headers around. This comes at a cost, though: we need to hide our implementation so far that we need to allow the user to not even know about the types we are using. In our first patch today, we’ll see how to do that.

Let’s first take a look at the changes we make to the Mechanism.h file, in the sspi library:

diff --git a/lib/sspi/Mechanism.h b/lib/sspi/Mechanism.h
index 804fef5..456c233 100644
--- a/lib/sspi/Mechanism.h
+++ b/lib/sspi/Mechanism.h
@@ -3,19 +3,25 @@
 
 #include "../security/Mechanism.h"
 #include "Details/prologue.h"
+#include <string>
 
 namespace Vlinder { namespace Chausette { namespace SSPI { 
 	class VLINDER_CHAUSETTE_SSPI_API Mechanism
 		: public Security::Mechanism
 	{
 	public :
-		Mechanism();
+		Mechanism(const std::string & package_name);
 		virtual ~Mechanism();
 
 	private :
 		// neither CopyConstructible nor Assignable
 		Mechanism(const Mechanism &);
 		Mechanism & operator=(const Mechanism &);
+
+	private :
+		struct Data_;
+
+		Data_ * data_;
 	};
 }}}

Like I said in a previous installment, there’s not much point in trying to hide the standard library implementation, as whoever uses our code will be using it anyway. There’s not much point, either, in trying to hide that an instance of Mechanism needs a package name to be created under SSPI: the MechanismFactory will do the creating for us, so if all goes well, the client code won’t even include this file. There is a chance, however, that they will, in which case we don’t want the user to be dependent on the Windows headers. That’s why I’ve created a forward-declaration to a private member structure called Data_. The structure itself is not here, but is defined in the .cpp file. It contains the package information we will need for later use, and it is conveniently hidden from view.

You might be wondering by now why the Data_ structure doesn’t have to be visible here – after all, I do have a pointer to an instance of it in my class. The thing is, though, that there are only a few reasons why the compiler needs to know the definition of a structure: it needs to know how to create an instance of it when one is defined or explicitly allocated with new; it needs to know how to destroy it when one goes out of scope or is explicitly destroyed with delete, and it needs to know how big it is when one is used as a member of something else. You might think that our use in this header would fall in that last category, but it doesn’t: our member is a pointer to an instance of Data_. It is not an instance of Data_ by itself. Pointers usually have the same size regardless of what they point to and, in any case, the compiler can figure out what that size is without knowing what it points to, so there’s no need, here, to know the definition of Data_ in order to use it.

diff --git a/lib/sspi/Mechanism.cpp b/lib/sspi/Mechanism.cpp
index 34fa65a..b4e7149 100644
--- a/lib/sspi/Mechanism.cpp
+++ b/lib/sspi/Mechanism.cpp
@@ -1,10 +1,40 @@
 #include "Mechanism.h"
+#include <windows.h>
+#include <security.h>
 
 namespace Vlinder { namespace Chausette { namespace SSPI { 
-	Mechanism::Mechanism()
-	{ /* no-op */ }
+	struct Mechanism::Data_
+	{
+		Data_(const std::string & package_name)
+			: package_name_(package_name)
+			, package_info_(0)
+		{ /* no-op */ }
+
+		~Data_()
+		{
+			FreeContextBuffer(package_info_);
+		}
+
+		std::string package_name_;
+		PSecPkgInfo package_info_;
+	};
+	
+	Mechanism::Mechanism(const std::string & package_name/* = std::string()*/)
+		: data_(new Data_(package_name))
+	{
+		if (QuerySecurityPackageInfo((LPSTR)package_name.c_str(), &data_->package_info_) != SEC_E_OK)
+		{
+			delete data_;
+			data_ = 0;
+			throw std::runtime_error("Failed to query package info");
+		}
+		else
+		{ /* all is well */ }
+	}
 
 	/*virtual */Mechanism::~Mechanism()
-	{ /* no-op */ }
+	{
+		delete data_;
+	}
 }}}

As you can see in this chunk, Data_ doesn’t do much of anything: it will free the PSecPkgInfo instance when it’s done, but that’s about all. All of the logic is still in the Mechanism class, where it belongs.

Now for the factory:

diff --git a/lib/sspi/MechanismFactory.cpp b/lib/sspi/MechanismFactory.cpp
index 8ffd315..3565a9c 100644
--- a/lib/sspi/MechanismFactory.cpp
+++ b/lib/sspi/MechanismFactory.cpp
@@ -1,6 +1,5 @@
 #include "MechanismFactory.h"
 #include "Mechanism.h"
-#define SECURITY_WIN32
 #include <windows.h>
 #include <security.h>
 
@@ -52,7 +51,7 @@ namespace Vlinder { namespace Chausette { namespace SSPI {
 
 	/*virtual */Mechanism * MechanismFactory::getDefaultMechanism() const/* = 0*/
 	{
-		return new Mechanism;
+		return new Mechanism(getAvailableMechanisms()[0]);
 	}
 
 	/*virtual */void MechanismFactory::releaseMechanism(Security::Mechanism * mechanism)/* = 0*/

As you can see, we assume that on Windows, the default package is whichever is listed first in the available packages – or “mechanisms” as we call them. This holds true for Windows and, as long as we don’t start sorting those packages, will hold true when we care for it to hold true.

Now a final little chunk:

diff --git a/tests/RFC1961/main.cpp b/tests/RFC1961/main.cpp
index 3d9540c..396f6ab 100644
--- a/tests/RFC1961/main.cpp
+++ b/tests/RFC1961/main.cpp
@@ -31,6 +31,7 @@ int main()
 	std::auto_ptr< Vlinder::Chausette::GSSAPI::MechanismFactory > mechanism_factory(new Vlinder::Chausette::GSSAPI::MechanismFactory);
 #endif
 	std::vector< std::string > mechanisms(Vlinder::Chausette::Security::MechanismFactory::getInstance().getAvailableMechanisms());
+	std::auto_ptr< Vlinder::Chausette::Security::Mechanism > mechanism(mechanism_factory->getDefaultMechanism());
 
 #ifdef _MSC_VER
 	WSADATA wsa_data;

in which we get the default mechanism in our test. Of course, we should really think of creating an actual unit test, but that’s beyond the scope of this installment.

So, data encapsulation works because we don’t need to know the definition of the data in order to be able to work with it, and it’s useful because it hides the nitty-gritty from those who don’t need to know it.

When RAII is too much

I’ve been going on about RAII in practically every installment of this podcast now – at least since I introduced the concept a while ago. You must either think that I am stark raving mad, or that I’m addicted to RAII. You might actually be right – whichever option you opt for – but you should also know that I can go without RAII for a few (albeit very few) lines of code. Sometimes, RAII is more trouble than it’s worth.

Look at the following chunk of code:

diff --git a/lib/sspi/Mechanism.cpp b/lib/sspi/Mechanism.cpp
index b4e7149..e801e14 100644
--- a/lib/sspi/Mechanism.cpp
+++ b/lib/sspi/Mechanism.cpp
@@ -1,6 +1,7 @@
 #include "Mechanism.h"
 #include <windows.h>
 #include <security.h>
+#include "Credentials.h"
 
 namespace Vlinder { namespace Chausette { namespace SSPI { 
 	struct Mechanism::Data_
@@ -36,5 +37,35 @@ namespace Vlinder { namespace Chausette { namespace SSPI {
 	{
 		delete data_;
 	}
+
+	/*virtual */std::auto_ptr< Security::Credentials > Mechanism::getCredentials(const std::string & principal, int flags)/* = 0*/
+	{
+		CredHandle cred_handle;
+		TimeStamp expiry;
+		if (AcquireCredentialsHandle(
+			(LPSTR)principal.c_str(),
+			data_->package_info_->Name, 
+			flags,
+			0 /* no LUID */,
+			0 /* no auth data */,
+			0 /* unused */,
+			0 /* unused */,
+			&cred_handle,
+			&expiry) != SEC_E_OK)
+		{
+			throw std::runtime_error("acquire credentials failed");
+		}
+		else
+		{ /* all is well */ }
+		try
+		{
+			return std::auto_ptr< Security::Credentials >(new Credentials(cred_handle));
+		}
+		catch (...)
+		{
+			FreeCredentialsHandle(&cred_handle);
+			throw;
+		}
+	}
 }}}

This is where I obtain a handle on the credentials of a given principal and, if that works, pass it on to a new instance of Credentials, which will manage its lifetime and whose lifetime is managed by an auto_ptr. Now, the important bit here is that there are two things that can go wrong between the try and the catch: the allocation might fail, in which case we’d be in big trouble but, more importantly, the constructor of the Credentials class would never be called, and construction of the Credentials instance might fail. If I were to define Credentials such that anything that is passed to its constructor will be taken over by the Credentials class, even if its construction fails – a popular and often worthy option – I’d have a problem in that I’d need to know which step went wrong. It is the option auto_ptr implements, though, and that is a Good Thing, though it would also complicate matters if construction of auto_ptr actually could fail.

Now, what would have happened had I used RAII for this task? There are two options: either I’d have created a scoped handle on the credentials handle, which I would have dismissed after the construction of the Credentials instance, but that would have left the same problem as I have now: determining which construction failed in order to know whether or not to dismiss the guard. An option that doesn’t actually make the code any cleaner is not really an option.

The alternative would have been to create a class that would explicitly hand ownership of the credentials handle to the new instance of the Credentials class, much like auto_ptr does with pointers. If it goes out of scope while still holding the reference, it will call the proper clean-up function.

Implementing that option can be fun – I would invite you to do it and contribute your result. It can also be far more work than writing a try-catch block with as its only benefit to make the code marginally cleaner. I say marginally because, although the clarity of the semantics would be a lot better, this piece of code isn’t repeated anywhere else. As such, weighing the benefits of implementing the “right” RAII solution (automatic ownership transfer) vs. the use of a try-catch block, the try-catch block wins because it is that much less work. Implementing the wrong RAII solution simply doesn’t make any sense.

So, I may be off my rocker and/or addicted to RAII, but apparently I still have a wee bit of common sense. I’ll try applying that again in four weeks – I’m taking the next two weeks off from the podcast, so there will be no mid-August installment.