XML - handle with care

I have been working with XML for about 5 years now and I would like to share my pain points with you. One big thing I have learnt about XML is that you cannot use it anywhere and everywhere, because sooner or later you are going to have an XML hell. Let me give you some examples.

When I created this website about a year back I did not want to use a heavy backend for it. I searched around for a lightweight database and did find some (I know you are asking for the names of those light weight databases but it's been a year now. I shall dig up some materials and add a separate post later). After not finding any good one that works well with .NET I decided to use XML. After all XML is everyone's favorite. Today as this site stands, it is running off of SQL Server. WHY?

• I invited a couple of friends to blog on the site, but as soon as I did that, I realized that I would have concurrency issues.
• I have added features like entries by author and entries by category. Adding those features with XML would be difficult. A lot of custom code would be required.

Second, I created a project about 3 years back where I stored a lot of information in XML files. Later due to load that application had to be run on multiple machines. Soon we were heading towards XML hell. Individual machines had a bunch of XML files. Anytime you had to search for anything you had to go to each and every machine and search the directory with *.xml.

So what is the moral of the story? Do not use XML? No, use XML, but plan ahead. Sometimes it is very difficult. Projects start small and everything works fine, until it gets bigger and bigger and things get out of hand. I am not saying XML is at fault. I am saying we need to be very careful of when and where to use XML.

In both the above examples when the project started, it started small, I never thought of inviting a bunch of friends to blog on my site. I realized the problem sooner, so I could fix the issue before it took my site down. Like wise for the other project I migrated all the data to SQL Server just like this web site and things started turning around.