Sunday, November 17, 2013

Using Log4net in C# - Quick start guide (and reminder to myself!)

I use logging in virtually every app I write. Even if it is just a simple proof of concept, or even if I'm tweaking someone else's code, the first thing I do is hook in log4net. This is a mature, lightweight logging framework which is easy to configure and easy to turn on and off.

And yet - I still do not remember all the steps needed to get it up and running, and hence this blog - a place for me and hopefully others to go to get log4net up and running so we can get onto more important tasks.

Installation and Setup

These days I usually use nuget to install log4net. You can also get it from Apache. It consists of one dll of 292kB.
The minimum steps needed to get it working are:

  • Reference the log4net.dll 
  • Add a line to AssemblyInfo.cs 
  • Declare a logger in each class 
  • Add logging statements 
  • Modify the .config file as needed - this is where the magic happens! 

You can also consult this very good setup summary by Justin Rhinesmith here

AssemblyInfo.cs

In AssemblyInfo.cs, add the line:

[assembly: log4net.Config.XmlConfigurator(ConfigFile="log4net.config", Watch = true)]

This tells your program to call the default configurator for log4net, use the file "log4net.config", and watch the file for changes. You could leave out the ConfigFile, in which case it would use the application's app.config file. I long ago decided that I would just use the same filename "log4net.config", for all my applications. Then I can simply copy over this file anytime I needed it. So far, this has worked out well and I tend to re-use the same settings 90% of the time.


Note that if you want to use the app.config file you still need to update AssemblyInfo.cs with:

[assembly: log4net.Config.XmlConfigurator(Watch = true)]

Also note that you can call the configurator explicitly in your code. I have found that this just does not work as well as declaring it in the assembly. In particular, declaring in the assembly means that you put that exact same line in assembly.cs for console, windows, ASP.NET and Service applications.

Using it in your code


Next, let's add logging to the code. Here is the basic code:

public class MyClass {
  static log4net.ILog log = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType); 

  public void Foo () {
    log.Warn("This is a Warning message");
    log.Info("This is an Info message");
    log.Debug("This is Debug message");
  }
}

GetLogger takes either a Type or a String. In this case, I am using reflection to get the type of "this" class.
I really like this way of doing it, since, it means that this same line can be cut and pasted into any class,
and will not need to be edited.

The log is also static, since I don't want to waste time and resources instantiating one in each instance of my class.

Next, I make calls to the logger, as in:

    log.Info("This is an Info message");

There you go. Now the code will send log information to log4net. So what happens to the information? Where does it go?  The very nice thing at this point is that where the log info goes is no longer in the application's hands. You don't  have to change any code to set or change where it goes.

Configuring log output


Log4net can be configured in the app.config file, within the code itself, or from any xml-formatted file. I like to just call my file log4net.config and be done with it.
My favorite log4net.config file can be found here.  I'm not including the text directly in the interest of saving space.  Feel free to copy it from GitHub!

This looks like any other config file for .Net. First a configuration handler is defined. Then comes the actual log4net configuration.  We add an appender, which can be defined as "a thing which takes log messages and puts them somewhere".  I will list my 3 essential appenders:  Trace, Console and RollingLog

TraceAppender

I work in Visual Studio, and so want  to see my log output in the Debug Output window.  The Trace Apender does that:

    <appender name="TraceAppender" type="log4net.Appender.TraceAppender" >
      <threshold value="DEBUG"/>
      <layout type="log4net.Layout.PatternLayout">
        <param name="ConversionPattern" value="%-5p %m%n" />
      </layout>
    </appender>

I generally want to see everything in this trace.

ConsoleAppender

The ConsoleAppender simply outputs messages to the console.  Many of my apps are console apps or have some form of console mode.   There is even a color-coded console appender if you want to get fancy.

 <appender name="ConsoleAppender" type="log4net.Appender.ConsoleAppender" >
      <threshold value="INFO"/>
      <layout type="log4net.Layout.PatternLayout">
        <param name="ConversionPattern" value="%-5p %m%n" />
      </layout>
 </appender>

I generally put the threshold to Info here, so that it does not show the verbose Debug messages.

RollingFile Appender

For persistent diagnostics, you want to log to a file. But what file? What happens when the file gets large? Log4net's file naming options take care of the first question, and the RollingFileAppender takes care of the second.

<appender name="RollingFileAppender" type="log4net.Appender.RollingFileAppender">
  <threshold value="DEBUG"/>
  <file type="log4net.Util.PatternString"
        value="%appdomain_%property{log4net:HostName}.log"/>
  <lockingModel type="log4net.Appender.FileAppender+MinimalLock" />
  <layout type="log4net.Layout.PatternLayout">
    <conversionPattern value="%date %-5level: %message%newline" />
  </layout>
  <rollingStyle value="Composite"/>
  <datePattern value="yyyyMMdd"/>
  <maxSizeRollBackups value="100"/>
  <maximumFileSize value="15MB"/>
</appender>

The file name is set in the <file> tag.  It basically gets the name of the executable itself and  the machine name.  So if my program is called foo.exe, running on Bombur, the file will be named "foo.exe_Bombur.log"  The file also rolls, meaning that if it exceeds a certain size, then it will rename the current log file and start a new one.  You can tell it how far to back up before deleting old log files, as well as how large to let those files get.  With the settings above, I generally have enough history to find recent issues, and the files are small enough to be read easily.

Log4net Debugging

But what if you don't see any messages? This is a concern about using a 3rd party logging system. If it does not work, how will you know?  The answer is that you can turn on log4net's own built-in debugging by putting the following into app.config:

<appSettings>
  <add key="log4net.Internal.Debug" value="false"/>
</appSettings>

<system.diagnostics>
  <trace autoflush="true">
    <listeners>
    <add name="textWriterTraceListener"
    type="System.Diagnostics.TextWriterTraceListener"
    initializeData="C:\log4net_internal.log"/>
    </listeners>
  </trace>
</system.diagnostics>

In the appsettings you tell log4net to run its internal debugging. If you run from the console you'll see many messages as log4net created repositories, adds appenders, and sets various properties. However, if you are debugging an ASP.NET application, you won't have a console. But the good news is that log4net's messages are emitted as trace calls. So you can add a tracelistener, such as a simple TextWriter in the example above. Now you'll see the messages in your log file.

This should be sufficient to find out what is going wrong.

However, if you still are getting no logs, then you can also just run from the debugger and step into the log4net code itself. It is all available in the download, and if you want to, you can even modify it yourself!

Summary

This was  intended to be a simple introduction to log4net, and a reminder to myself of the basocs.  I hope it helps to show how simple it is to use, and how easy it is to add the benefit of logging to your development experience.

Thursday, March 21, 2013

Primavera P6 Database Install and Case (in)sensitive Passwords in Oracle 11g

Primavera P6 Database Install and Case (in)sensitive Passwords in Oracle 11g

Recently while at a client we were running through the P6 installer to create a new P6 database on Oracle 11g.  This usually goes smoothly, as long as all of the connection information is entered correctly.  But this time we hit a snag.
We entered our connection information had the dba enter the secret admin password. Then we got an error:



SQLException: ORA-01017: invalid username/password; logon denied

What's going wrong?  It seemed we were entering everything correctly, but it simply would not connect.

First, we verified that we could ping the database server.  Yep, worked fine.
Next, we ran tnsping to see if we could see the database.  Yep, that worked too.
Then we ran SqlPlus from the command line, using the same server/SID and same user and password.  Again, everything worked fine.

We used the same connection information in the P6 setup dialog as what we verified with SqlPlus.  This was quite a mystery.  Now it's not uncommon for someone to fat-finger the password, or to mis-type the server or SID.  But we had two people working on it and double checked, triple checked, nothing!

Then we asked around at the client if anyone had seen this kind of behavior before.  Sure enough, someone asked if the system password was all uppercase or not.  No, it certainly was not, because who would ever have a password with only uppercase characters?  Well, it turns out that until 11g, oracle database passwords were ALWAYS stored as uppercase.  I did not know that.

Usually you would not know this because the various components that connect to an Oracle database know to convert the password you enter into uppercase before sending it to Oracle, and silently do so. And this is exactly what was happening.  The components used by dbSetup for the underlying database connection were taking the password we entered, translating it to uppercase, and that password was being rejected by the server.

Our immediate workaround was to have the dba change the password to all uppercase.  A terrible solution, but we were pressed for time, and just needed that connection once to create the database.

In the light of the next day, I found a better solution.  Oracle allows you to wrap your password in double quotes, which causes the password to be passed directly, with no translation to uppercase.  This is the better solution by far.

So - why did this catch us unawares?  Well, this is a new feature in 11g, and the case-sensitivity must be explicitly turned on.  The XE client is still 10g, so it's not possible to have this issue during routine laptop installs of P6.  And our clients who have recently upgraded to 11g have not enabled this feature yet.

A really good article explaining the new case-sensitive password feature in 11g can be found here:

http://www.oradba.ch/2011/02/case-sensitive-passwords-and-strong-user-authentication-2/


Well, another day, another lesson learned.  And the overall lesson is that with software you just never know what's coming around the corner.  Even though we install Primavera P6 routinely, when we get to a client it is rarely routine.  Certain parts get smoother every time, but you never can tell what differences you'll find until you attempt the install on a specific system.


Wednesday, February 27, 2013

Timberline Address Book bug Introduced with 2012 year-end update to Accounting 9.8

Timberline Address Book bug Introduced with 2012 year-end update to Accounting 9.8

In December 2012 one of our Timberline/Primavera integrations customers applied the year-end update to Timberline 9.8 Accounting system.  Over the next few weeks they noticed that some emails on Address Book companies were blank - even though they had been set in the past.  It was hard to notice, and no pattern was visible.
Just this week (Feb 26, 2013), our client found out what was happening.  It is a bug introduced with that year-end update.

The bug is related to the new ability to add an email address to an AP vendor.  In AP there are now two email fields, as you can see in the screens below.

In the old AP, there was just Contact 1 and Contact 2, and each had a  Name, Telephone and Extension (left image).
In the new AP there is now an Email as well (right image).


The issue is that this email on Contact 1 on the AP Vendor  is strongly tied to the email on the Address Book (AB) Company for that vendor.  You can see the AP Vendor that the AB Company is associated with (if any) by opening up the AB Company and viewing the "Use As" link.  If there is an AP Vendor, it will be checked and the number will be shown in parentheses.



If a change is made to the AP Vendor, then that email - and just that email - overwrites whatever email has been set on the AB Company.

Here is the major problem - since that AP Vendor email field is new, it is always blank.  This means that if you update your AP Vendor in any way, you are also overwriting the AB email to be blank!  If the AP Vendor email is set to a specific address, that address too will overwrite the AB email.

Since we have clients who use Timberline regularly, and who rely on these email addresses being correct,  we have a fix for this, which should serve as a patch until Timberline fixes it themselves.

This fix is part of our Dimension Integration framework.  Since we have many integrations which work with Timberline, we were able to quickly develop this fix.  This integration module does the following:

It searches all AB Companies, and finds their matching AP Vendors, if any.  Then if the AB Company email is not blank and the AP Vendor email is blank, it updates the AP Vendor with the AB Company email.  This way, when someone updates the AP Vendor, the value being written back to AB is the correct value.

Of course this is just a stop-gap solution.  You should be able to enter one email for AP, and another for AB, and the two should not be tied together like this.  But until this is fixed, at least our clients can move ahead.



Sunday, February 3, 2013

Primavera P6 Professional Client and 8.2 EPPM

I was recently at the Construction CPM Conference in New Orleans.  While there I had many great conversations with people about Primavera P6. Some people were new to P6, though  most were long-time experts in it, but using versions other than 8.2.  There is something fundamentally important that happened in P6 version 8.2 that many people were not aware of, and I thought it important to clear this up:

The desktop client for 8.2 can be used to connect to a Professional or Enterprise (EPPM) instance of P6.


Before showing how this is done, let's straighten out some terminology.  First there is the "application" of P6 that you are using.  This can be EPPM or Professional.  They are both fundamentally P6, and both are on version 8.2 now.  But they are installed differently, and intended for different users and situations.

P6 Professional is designed to be used by a single user or a small team.  It is the version that I often run on my laptop when I'm trying out ideas and have no connection to the internet.  There are two parts of this: a database, and a Windows desktop program called the P6 Professional Client.  The database runs on your laptop, or on a server on the network.  The only way to interact with P6 is to use the Professional Client.  On your Windows 7 Start menu, it looks like this:



P6 Professional can be considered a "lightweight" or even "portable" version of Primavera P6. As I mentioned, it works great on laptops, and for single users is a great tool.


EPPM is the Enterprise version of P6.  It is designed to be used by many people on large teams, though it is used by small teams as well.  This is the "flagship" version of P6, the one where Oracle is focusing its innovative efforts. You can interact with EPPM in a number of ways:

  • Web browser (Web Client)
  • Team Member
  • Progress Reporter
  • Web Services
  • P6 Professional Client


It has features that are not in Profesional, such as the Risk Matrix, Portfolio Views, Dashboards, and much more.  When many people are first introduced to EPPM, they hear about all these features, all these ways to interact with it, and often they are told that you can only use the Web Client. And this is simply not true.  You can use P6 Professional Client.  I myself switch between the Web Client and the Professional Client often.  There are simply some things that I am simply used to doing in the Professional client that I feel are either easier, or more responsive there.  For example, I will often create a new schedule or template in Professional.  But when viewing or making small changes to a project, the Web Client is simply easier to get to.

One source of confusion about the use of the P6 Professional versus the Web Client  is what happened with version 8.0 and 8.1  In those versions, there were two different programs which looked and acted like the P6 Professional Client.  The one for EPPM was called the Optional Client, and though it looked like Professional, it was a different program, and could only connect to EPPM databases.

With 8.2, that is all gone.  The same P6 Professional you use to connect to a local instance on your laptop is the same P6 Professional you use to connect to your corporate EPPM instance.  You have different connection settings, and most likely a different set or logon credentials, but the program is the same.


Now let's get back to the example.  Below you can see that on my desktop I have a few P6 database instances running on SQLServer Express:



And I have made two Database aliases in P6 Professional


P6 EPPM Empty Bombur - connects to a P6 8.2 EPPM database
P6 Pro Empty Bombur - connects to a  P6 8.2 Professional database


I can connect to the EPPM instance, and get the screen below:



There are two things to notice:

No Admin Menu  When connected to EPPM, the administrative functions have been moved into the Web Client, and so the Admin menu, which normally appears between Tools and Help simply does not appear.

(EPPM) shown on the bottom In the lower right-hand corner of the screen the connection alias "P6 EPPM Empty Bombur" is followed by "EPPM" in parentheses. This tells me that I have connected to an EPPM database.

Next we use the exact same program to connect to a P6 Professional instance and get the screen below:

Here are the differences:

Admin menu is shown.  Since P6 Professional is designed to be used by individuals, or small teams, each user can have the ability to perform administrative actions, such as adding new users.

(Professional) shown on the bottom  Now in the lower right corner, the word following the database alias is "Professional".

There you go, evidence that with Primavera P6 version 8.2, you can use the Professional Client to connect to a full-blown EPPM system sitting behind your coroprate firewall, or a lightweight instance of P6 Professional sitting in a local database.  I hope this clears this up, and perhaps will encourage those using earlier version of P6 to upgrade to P6 8.2. It certainly has made my life easier!

More information on P6 8.2,  can be found in my book, "Oracle Primavera P6 Version 8" by Packt Publishing.