This is not a new challenge. But "maintainability", which encompasses all the things stated above, is an important application characteristic in today's world. Every IT department has a say when it comes to acquiring new enterprise applications. And besides performance and scalability, they will inquire in more and more detail about how easy it is to maintain your application, how easy it is to get proactively notified about application issues with suggested resolutions, and how easy it is to troubleshoot your application.

Windows Management Instrumentation to the rescue

WBEM itself uses a standardized data model to describe manageable objects like systems, applications, services, networks, etc. This is the Common Information Model (CIM) and you can find more information about it here. The WMI data model is extending the CIM data model. WMI is a component of the Windows OS and is included out-of-the-box on Windows 2000, Windows XP and Windows 2003. It is located in the "windows\system32\wbem" folder. It can be downloaded for earlier versions of Windows from here (with a number of other WMI tools) . WMI also provides a rich programming interface including VB, C++, scripting languages and the .NET framework itself.

A WMI architecture overview

Layer two is the WMI infrastructure which is provided by the Windows OS itself. The WMI infrastructure consists of the "Windows Management Instrumentation" service and the WMI repository (store). The Windows Management Instrumentation service is the intermediary between the providers, the consumers and the WMI store. Providers can place information through it into the WMI repository, consumers can query the WMI store for available information, and the WMI service can pass information directly between providers and consumers.

Layer three consists of the management applications, the applications which are interacting with WMI to find information about managed objects. Management applications gather information to analyze it and provide a comprehensive and accurate view of what is happening to these managed objects. Operators can use this raw or processed information to make decisions, take actions and resolve items. Microsoft's implementation of such a management application is Microsoft Operations Manager 2005 (which we cover later in the article).

A look at the WMI data model

This is exactly the same as in the .NET framework with the exception that you define these WMI classes through WMI and store them in the WMI repository. You can group classes together into namespaces, like your namespaces in .NET. Each provider typically creates its own namespace, for example "root\enterpriseminds". The root namespace is called "Root" and all other namespaces are children of it. Each namespace can have one or multiple classes and child namespaces. The picture on the left shows a sample (please note this is an abbreviated view using the sample application attached to the second part of this article to demonstrate this namespace and class hierarchy). On top you see the Root namespace which has two classes. System classes always start with two underscores. The Root namespace has aspnet, cimv2, default and directory child namespaces. In the picture you can see that the cimv2 namespace has two system classes followed with a number of Win32 classes implemented by it. The class Win32_DefragAnalysis, for example, provides a defrag analysis and returns all the information back to the WMI consumer. The namespace cimv2 is where you find all classes provided by Windows itself. All classes provided by IIS are in the microsoftiisv2 namespace. You can create your own namespaces as needed, e.g. root\mycompany.

 Like in .NET, you cannot instantiate abstract classes. All other classes can have class instances, actual objects which hold data (information). Class instances can be static or dynamic. Static class instances are stored in the WMI repository and the WMI service queries it directly from the WMI store. Dynamic class instances are provided in real time by underlying providers. Classes which provide dynamic class instances are marked with the Dynamic qualifier and also have a Provider qualifier which specifies the name of the provider. Providers (or rather, the information about providers) are stored in the WMI store as static class instances of the class __Win32Provider. The provider named in a Provider qualifier needs to have a __Win32Provider class instance so WMI knows the class ID and other related information it needs to communicate with the dynamic provider. Almost all classes defined in the cimv2 and microsoftiisv2 namespace are dynamic.

WMI provides class instances (objects) and events. Both are defined as WMI classes in the WMI repository. The difference is that consumers have to actively query for class instances while they can subscribe to events; as soon as an event is available the consumer gets notified about it. More on that later in this article.

The .NET framework support for WMI

• WMI classes defined through managed providers cannot implement methods or qualifiers. However, it is possible to call methods of classes defined through unmanaged code.
• Properties defined through managed providers are read-only for consumers.
• WMI classes defined through managed providers are always marked with the Dynamic qualifier.

Hopefully future versions of the .NET framework will remove those limitations. But the .NET framework makes it very easy to define WMI classes, raise WMI events and provide dynamic WMI class instances.

A look at the System.Management.Instrumentation namespace

[assembly: Instrumented("root/enterpriseminds")]

Specify the namespace hierarchy your WMI classes and events will appear in. The .NET framework will create this namespace for you when registering the WMI classes in the WMI store. Always start with the root namespace and use the forward slash and not the backward slash as WMI itself does. Next define the WMI classes and WMI events you want to expose. You can do this through attributes or by inheriting from the Instance or BaseEvent classes provided by .NET. Here is how you define a WMI class using attributes:

[InstrumentationClass(InstrumentationType.Instance)]
public class InfoMessage
{
     public string Message;
     public string Guid;
     public int Type;
}

First define your .NET class (you can also use a struct) and then apply the attribute InstrumentationClass specifying the InstrumentationType as Instance, meaning you provide class instances. You can only use simple data types, or complex data types you also register in the WMI store. For example, you cannot use an enumeration as data type (because it is unknown to WMI), and you cannot register an enumeration. But you can create your own MessageType class which you then also register with WMI through the InstrumentationClass attribute. Inheriting your class from the Instance class provided by .NET achieves the same as using the InstrumentationClass attribute.

public class InfoMessage : Instance
{
     public string Message;
     public string Guid;
     public int Type;
} 

Every public property and public field becomes a WMI class property. Private properties, private fields, and all methods are ignored. The class also gets assigned the Dynamic and Provider qualifiers. The .NET framework uses its own WMI classes for this purpose - WMINET_InstrumentedAssembly and WMINET_ManagedAssemblyProvider. Whenever your WMI class gets registered in the WMI store (which happens every time the WMI class to register changes) it creates a new WMINET_InstrumentedAssembly and WMINET_ManagedAssemblyProvider class instance and sets the value of the Provider qualifier of your class to the value of WMINET_ManagedAssemblyProvider.Name. This information tells the WMI service and .NET framework how to find the dynamic provider, which consists of your assembly and types representing this WMI class.

You can apply the IgnoreMemberAttribute to any public property or public field you don't want to be registered as property in the WMI class. To provide an instance of the WMI class to consumers you create an instance of your .NET class, set its values and then call Instrumentation.Publish() to publish the class instance. From then on the object is visible to any WMI consumer.

// create a new message
InfoMessage Message = new InfoMessage();

// set its properties
Message.Message = MessageText;
Message.Guid = Guid.NewGuid().ToString();
Message.Type = (int)Type;

try
{
      // publishes the message to the WMI repository
      Instrumentation.Publish(Message);
}

// we have not been able to publish the message
catch(ManagementException)
{
}

If you inherited your class from the Instance class instead of using the attribute then you have a Published property. Setting it to true publishes the class instance. WMI classes created through managed providers are dynamic, meaning that, when your application domain gets destroyed and your assembly unloaded, any class instance you published gets revoked automatically. You can revoke any class instance you published by calling Instrumentation.Revoke() and passing along the same class instance (or for classes inherited from the Instance class you set the Published property to false).

WMI classes are your choice if you want to publish information so consumers can query it and get information about your application or assembly. WMI events are your choice if you want to proactively notify consumers about something. There is not much difference in declaring and firing events. You define again your .NET class or struct and apply the InstrumentationClass, but this time you set the InstrumentationType to Event.

[InstrumentationClass(InstrumentationType.Event)]
public class EventDetails
{
      public string Message;
      public string Guid;
      public int Type;
}

The same limitations apply as to WMI classes. Only simple data types or complex types which are also registered as WMI classes can be used. You can also inherit the class from BaseEvent instead of applying the attribute.

public class EventDetails : BaseEvent
{
      public string Message;
      public string Guid;
      public int Type;
}

Events are not published and revoked like class instances. Events are fired at a given time and any consumer listening to this event type will receive it. But any consumer subscribing to this event type afterwards will not get the event. It will only get future events fired. Here is a code snippet which shows how to fire an event:

// create a new event
EventDetails Details = new EventDetails();

// set the event details
Details.Message = Message;
Details.Guid = Guid.NewGuid().ToString();
Details.Type = (int)Type;

try
{
      // fire the event so consumers can consume it
      Details.Fire();
}

// catch any exception and return false
catch (ManagementException)
{
}

You first create your .NET class instance, set its properties, and then call the Fire method (if inherited from the BaseEvent class) or call Instrumentation.Fire() passing along the .NET object (if you used the attribute to declare your WMI event). This is all you need to do to publish WMI class instances and fire WMI events. Add a .NET installer so that you can manually or even automatically register the WMI class and event in the WMI store when first loading the assembly. Here is the code snippet:

[RunInstaller(true)]
public class MyProviderInstaller : DefaultManagementProjectInstaller
{
}

You can manually run InstallUtil against this assembly to perform the registration in the WMI store. It really can't get easier to instrument your application.

A simple WMI consumer

\<machinename>\root\<namespacename>:<classpath>

You can use a dot instead of the machine name to point to the local machine. For example, to connect to the WMI class created in the previous section, use the path "\.\root\enterpriseminds:InfoMessage". Through the method GetInstances() you get a collection of all class instances of this class. Class instances are represented by the ManagementObject type. The ManagementObject provides a property collection to access all the properties defined by the class. The following code snippet queries for all available class instances of the InfoMessage class:

ManagementClass WMIClass = null;

try
{
      // bind to the WMI class
      WMIClass = new ManagementClass(@"\.\root\enterpriseminds\InfoMessage");

      // now loop through all the object instances
      foreach (ManagementObject Object in WMIClass.GetInstances())
      {
            // create a new message
            InfoMessage Message = new InfoMessage();

            // read the properties out of the object instance
            Message.Message = Convert.ToString(Object.Properties["Message"].Value);
            Message.Type = Convert.ToInt16(Object.Properties["Type"].Value);
            Message.Guid = Convert.ToString(Object.Properties["Guid"].Value);

            // add the new message to the list view
            ItemCollection.Add(new ListViewItem(new string[] { Message.Message,
                               Message.Type.ToString(), Message.Guid }));

            // release the underlying COM object
            Object.Dispose();
      }
}

// any exception happening is shown
catch (Exception e)
{
      MessageBox.Show(e.Message);
}

// release the underlying COM object
finally
{
      if (WMIClass != null)
            WMIClass.Dispose();
}

First we bind to the WMI class, then we loop through the collection of class instances we obtain by calling GetInstances() on the ManagementClass type and for each class instance we read from the property collection the three properties Message, Type and Guid. Each class instance we find gets added to a list view. The .NET WMI types are built on top of the unmanaged WMI API. Therefore you should call Dispose() on each ManagementObject you get from GetInstances() as well as on the ManagementClass object itself. That is all you need to do to get the class instances of any WMI class.

You can perform WQL queries against the WMI store. WQL is built on top of the SQL language with some variations, the main difference being that you can only query for data and not perform updates or deletes. A separate section of this article is dedicated to WQL. To subscribe to WMI events you use the WqlEventQuery type which creates the appropriate WQL query for you. When you create an instance of WqlEventQuery you pass along the event class name to query for and an optional condition, like any SQL WHERE condition. The condition allows you to query for only specific events, for example all EventDetails events where the Type property is set to one InstrumentationConsumer Consumer = null;
WqlEventQuery EventQuery = null;
ManagementScope Scope = null;

try
{
     // create a event query object
     EventQuery = new WqlEventQuery("EventDetails");

     // define the search scope - which machine and namespace
     Scope = new ManagementScope(@"\.\root\enterpriseminds\EventDetails");

     // create a watcher object we use to get async notifications
     EventWatcher = new ManagementEventWatcher(Scope, EventQuery);

     // register the delegate to call when an event arrived
     EventWatcher.EventArrived += new EventArrivedEventHandler(Delegate_EventArrived);

     // start listening for events
     EventWatcher.Start();
}

// show any exception happening
catch (Exception e)
{
      MessageBox.Show(e.Message);
}

This code snippet subscribes to EventDetails events in the "\.root\enterpriseminds" namespace. To synchronously wait for the next WMI event call WaitForNextEvent() on the ManagementEventWatcher object. This call waits till the next instance of that event is available and then returns. The code snippet above performs an asynchronous read by setting the event handler to be called when an event arrives and then calling Start() on the ManagementEventWatcher object. This allows your application to get notified about WMI events while performing other tasks. The next code snippet shows an example of the EventArrived event handler (of the type EventArrivedEventHandler):

public void Delegate_EventArrived(object sender, EventArrivedEventArgs e)
{
      // create a new event info
      EventDetails Details = new EventDetails();

      // get the event details
      Details.Message = Convert.ToString(e.NewEvent.Properties["Message"].Value);
      Details.Type =    Convert.ToInt16(e.NewEvent.Properties["Type"].Value);
      Details.Guid = Convert.ToString(e.NewEvent.Properties["Guid"].Value);

      // now add the event to the list view
      EventCollection.Add(new ListViewItem(new string[] { Details.Message,
                          Details.Type.ToString(), Details.Guid }));
}

The EventArrivedEventArgs.NewEvent contains a reference to the received WMI event and is of the type ManagementBaseObject. It provides access to the property collection of the WMI event which we use to read the event details and then we add the event to a list view. Finally if you do not want to receive any further events you call Stop() on the ManagementEventWatcher object and then Dispose() to free the underlying COM object. Here is a code snippet:

// stop the event listening
EventWatcher.Stop();

// and dispose the underlying COM object
EventWatcher.Dispose();

It is also very simple to subscribe and asynchronously receive WMI events. It is also very powerful to perform that against any machine on your network which has the WMI infrastructure installed, which all new Windows releases do. A good example would be a server application which you deploy onto multiple servers. How do you monitor it and make sure you get notified as soon as a critical error happens? With WMI you can achieve that fairly easily. When a critical error happens, you raise a WMI event. You then write a small application which subscribes to this WMI event on all the machines this server application is running on. And as soon as one raises an event, you can display it on a central console. This simple management tool could be very powerful and helpful for monitoring your server application. Microsoft provides such a management tool, just much more sophisticated Operations Manager 2005.

The attached sample application

A look at Microsoft Operations Manager 2005

To install MOM 2005 launch setup and first perform the option "check prerequisite". Select all the items under "Microsoft Operations Manager 2005 components" and click Check. This will check that all the pre-requisites like Windows 2000 SP4 or Windows 2003, MDAC 2.8, SQL Server 2003 SP3a and the .NET framework 1.1 are installed on the machine. After all the pre-requisites are installed, go back to the main setup screen and select the option "Install Microsoft Operations Manager 2005". Select the typical installation which will again perform a pre-requisite check. The check will pass as we already made sure that all pre-requisites are installed. Next select the database server to use followed by the database size which needs to be at a minimum 300 MB. If you have multiple management servers running, then you can group them in one or multiple groups. The group name cannot be changed after the installation. Next you enter the group name to use, for example "Development". Next you enter the Windows account to be used by any action performed by this management server, for example to gather information from providers, install and uninstall agents, etc. Next you enter the Windows account to be used to access the MOM database server. Finally it asks whether to enable error reporting and whether to send them automatically to Microsoft - always a good idea. When the install is completed you find a new start menu item under "Programs | Microsoft Operations Manager 2005".

MOM 2005 consists of an "Administrator Console" and an "Operator Console". This separates very well the administrative tasks from the operational tasks. Administrative tasks are installing agents at new servers to monitor, setting up new event providers like a WMI event, setting up alert rules like "email or page an operator", etc. Operative tasks are looking at the health of monitored servers, reacting to alerts, analyzing event and alert information, etc. So Operators are the IT professionals who use MOM 2005 as a tool to look after the server and IT infrastructure they are responsible for. First launch the "Administrator Console" to set up MOM to receive the WMI events we are raising in our application. This shows the console tree on the left side and the "Information Center" on the right side. The information center gives you access to documentation, MOM downloads, the MOM home page, etc.

MOM utilizes so called Management Packs, which are a set of rules, providers, scripts, tasks, etc. A Management Pack encapsulates all the management aspects to monitor an application or infrastructure component. You can say it encapsulates all the knowledge to manage that application or component. The ISV of that application or component should provide the Management Pack, as the creators also know best how to monitor it and keep it running reliably. Microsoft internally mandates that all future server products/releases shipped need to provide a Management Pack. MOM installs out of the box with the "Microsoft Operations Manager 2005" Management Pack but ships with a number of other Management Packs which you can import, for example for SQL Server, Active Directory, Systems Management Server, IIS, DNS, etc. To import a Management Pack, click on the item "Management Pack" in the console tree and select "Import/Export Management Pack" from the popup menu. Next select that you want to import a Management Pack followed by the folder where the Management Pack is located. Your MOM 2005 CD has a folder called "ManagementPacks". Browse to it and on the next screen you find all the Management Packs available in that location. Select one or more Management Packs to import, for example "Microsoft SQL Server 2000" and "Microsoft Windows IIS". During the import it shows you a detailed progress screen. When done you will see new items appearing under the "Management Packs" item in the console tree. For example under "Rule and Groups" you see "Microsoft Windows Internet Information Services" and "Microsoft SQL Server". Underneath you find detailed event and alert rules which are recommended by those Microsoft product groups.

Creating your own Management Pack

Expand the newly created "Rule Group" with the plus sign in front of it. Next create a new event rule (events meaning MOM events collected). Right click on the item "Event Rule", select "Create Event Rule" from the popup menu and from the list select "Alert on or Respond to Event (Event)". First you select an existing event provider or create a new one. Create a new event provider by clicking on the button "New". The provider type we use is WMI, so select "WMI Events" from the list. Next enter the name of the new provider, for example "Enterprise-Minds WMI Event". Then you enter the namespace the WMI class is residing in which is, in our case, "root\enterpriseminds". Next you enter the WQL query this provider executes against WMI, which in this case is "SELECT * FROM EventDetails". To query only for certain properties, list the properties instead of the star, for example "SELECT Message, Type FROM EventDetails". This query returns any WMI event raised by the sample application. Another way to only return certain properties is to list the properties in the "Property list" field, for example "Message, Type". Click "finish" to create this new provider, which brings you back to the "Event Rule" creation process and selects the new provider in the drop down list.

On the next screen you could apply a filter if you don't want to receive all events, for example only receive events where the Type property has a value of one. Later in the article it is explained how to filter events based on the WMI event values. Next you could define that this rule only runs on certain days and times, the default being "process at any time". Next you can generate an alert by selecting the "Generate Alert" checkbox. Select a severity of the alert which is by default "Critical Error". Later in the article it is explained how you can generate different alerts based on the values provided by the WMI event. Next you select if you want to suppress duplicate alerts, which is the default selection, and which criteria are used to determine whether it is a duplicate alert. The default criterion used is if the alert comes from the same computer and domain. This reduces the amount of alerts an Operator has to deal with, as it is very likely that the same alert will be raised many times till the issue is resolved. It is enough that the Operator sees one alert which keeps the noise level down. Next you define response actions which should be executed when the event happens, for example execute this script which may restart an application. This allows automating some of the actions to be taken when this event happens. Next you enter a knowledgebase which allows you to describe the issue in detail and standard troubleshooting steps to be taken. This helps Operators to understand how to resolve an issue. Finally you give the event rule a name, for example "Instrumentation Sample Rule".

This is all you need to do to capture WMI events and raise alerts to Operators. In order for all the changes to take effect you need to right click on the item "Management Packs" and select "Commit Configuration Change" from the popup menu. By default, changes are only committed and take effect after five minutes. With this you can commit the changes right away. To change this default setting, select the item "Administrator | Global Settings". Then, in the list on the right side, select "Management Servers", double click the item you want to change and, on the "Rule Change Polling" tab of the dialog that comes up, change the time till changes are committed.

The IT Operator view in MOM 2005

Next launch the "InstrumentationProviderForm" sample application attached to this article. Launching it will raise an EventDetails WMI event which MOM 2005 is now listening for. In the same application you can enter the event text, select the event type and then click the "Fire event" button to raise additional WMI events. Go back to the "Operator Console" and to the alerts view. You will see now an alert generated because of the WMI event fired by the sample application. Click on the alert and you will see its details in the lower "Alert Details" pane. In the alert details you see all the WMI event properties MOM received. You see first the system properties like __PATH (the WMI event path), __SUPERCLASS (the class this WMI event inherited from), etc. You then see also all the WMI event properties itself. Please take special note of the order they appear in. Later in the article when we set event filters and alert severity, you can select the properties by number. For example, the __PATH property is the fifth one so you can chose it by the name Parameter 5, the Message property is Parameter 10 and the Type property is Parameter 11. Also take special note of the value of the Type property. The property is of type integer in the .NET code but it is shown here as "0 (0x0)", "1 (0x1)", etc. So MOM reads the numeric value and converts it to a string which has in parenthesis the hexadecimal string representation of the value.

If you select the Event tab in the "Alert Details" pane then you see all the events which raised this alert. In the alert details there is a wealth of information available, such as when the alert was first and last raised. When the issue has been resolved, the Operator right clicks on the alert, selects "Set Alert Resolution State | Resolved" and then enters a description how it was resolved for future reference. Through that menu Operators can also set other alert statuses, such as that it has been assigned to a vendor or subject matter expert, or it might require a scheduled maintenance. When all alerts have been resolved then the health state is set back to "Success".

Filtering MOM events and setting alert severity levels

Save the event rule and don't forget to commit the configuration changes (right click on Management Packs and select "Commit Configuration Change" from the popup menu). Go back to the sample application and manually raise three events, setting the Type for each to "Application Start", "Application Stop" and "User Initiated". Go back to the Operator console and refresh the Events view (key F5) and you see only two events. Select each and check the value of the Type property and you will see that the one with the value "2 (0x2)" (which is "User Initiated") is not present, because it has been filtered. Switch to the alert view and you will see only one alert, because we suppress duplicates. Select the alert and select the Events tab in the "Alert Details" pane and you see the two events which raised the alert. Set the state of the alert to resolved so we have no alerts for our next scenario.

Now this time we want to get all events but for different values in the Type property we want to raise alerts with different severity. Bring up again the properties of the "Instrumentation Sample Rule" rule. First go to the Criteria tab and remove the filter (click on the Advanced button, select the criterion in the list and click the Remove button). Next go to the Alert tab and select the check box "Enable state alert properties". This allows us to create a complex rule which defines the severity on property values. Click on the Edit button beside the "Alert severity". Click Edit to edit the default condition added by MOM. The new dialog allows you to create a complex condition, and the severity level if that condition is met. Select Parameter 11 through the Parameter button on the left side. You see that it adds "AttributeValue(Parameter 11)" to the condition text box. Edit it so it says "AttributeValue(Parameter 11) = "0 (0x0)"" and set the alert severity to "Information" through the drop down list at the bottom. This condition says if the event Type property is set to "Application Start" then raise an information alert. Go back to the condition dialog and click on the Add button to add another condition. Select again Parameter 11, set the condition to "AttributeValue(Parameter 11) = "1 (0x1)"" and set the severity level to "Critical Error". This condition says when the event Type property is "Application Stop" then raise a "critical error" alert. Repeat the same steps to add a condition "AttributeValue(Parameter 11) = "2 (0x2)"" where the severity is set to warning. This condition says when the event Type property is "User Initiated" then raise a warning alert. Close the condition dialog, which brings you back to the alert tab on the event rule. We are raising now three different alerts, which means you also need to change the criteria used by MOM to suppress duplicate alerts. Set the "Server role" to "MOM Agent", the "Component" to "WMI" and for the Instance you select (through the arrow on the right) "Source Computer" and then "Parameter 11". This sets the Instance value to "$Source Computer$$Parameter 11$". So only duplicate alerts coming from the same machine with the same Parameter 11 value (which is the event Type property) are suppressed.

Commit the configuration changes and go back to the sample application. Manually raise three events again, setting the Type for each to "Application Start", "Application Stop" and "User Initiated". Go back to the Operator Console and you will see again three events, but you also see three different alerts. One with the severity "information", one with the severity "warning" and one with the severity "critical error". If you raise multiple events through the sample application you will also see that only one alert per severity is raised.

Creating your Management Pack

Download Source

Summary

The second part of this article will dive deeper into the WMI store, classes and class instances. WMI can provide a wealth of information about a machine. The sample application for the second article will provide a "WMI System Browser" which enables you to find Windows and hardware serial numbers, a list of installed OS hotfixes, a list of processors with clock speed and current processor status information, and much more. The article will cover how to programmatically create and edit class and class instance information. If you have comments to this article, please contact me @ klaus_salchner@hotmail.com . I want to hear if you learned something new. Contact me if you have questions about this topic or article.

About the author

Klaus calls Vancouver, British Columbia his home at the moment. His next big goal is doing the New York marathon in 2005. Klaus is interested in guest speaking opportunities or as an author for .NET magazines or Web sites. He can be contacted at klaus_salchner@hotmail.com or http://www.enterprise-minds.com .

Enterprise application architecture and design consulting services are available. If you want to hear more about it contact me! Involve me in your projects and I will make a difference for you. Contact me if you have an idea for an article or research project. Also conact me if you want to co-author an article or join future research projects!

Edited by Dwight Johnson