- Posted by matt on October 21, 2008
In my ongoing efforts to create a helper library for myself that consists of commonly used functions in my development efforts, I’ve decided to write a sitemap generator. The concept is simple, I just want the contents of my Web.sitemap file to produce a Sitemap.aspx page which will serve as the basic sitemap for a web site.
I’ve found that on every new web site I’m building in .NET, I need a sitemap for one reason or another. Many times the reason is as simple as making it easy for whomever is writing the content to see what pages still need content and to quickly navigate through the site. It also doesn’t hurt to leave a sitemap link in your sites footer to help out your users and/or search engines.
Requirements
The requirements are simple. Add a .sitemap file to the root of your web site and fill it with the structure of your site. At the minimum you need a valid xml document and a root siteMapNode. Ideally, you would put your entire site structure into the sitemap. For example:
Then you can use the following class in your project:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Web;
namespace SevenLabs.Utilities.Web
{
public class Helper
{
#region Sitemap
/// <summary>
/// Builds out the sitemap into html using lists and linking to each page.
/// </summary>
/// <param name="rootNode"></param>
/// <returns>A string holding the html</returns>
public static string BuildSitemapHtml(SiteMapNode rootNode)
{
StringBuilder sb = new StringBuilder();
if (rootNode != null)
{
sb.AppendFormat("<h1><a href='{0}' runat='server'>{1}</a></h1>", rootNode.Url, rootNode.Title);
if (rootNode.HasChildNodes)
{
sb.Append("<ul>");
sb = BuildSitemapChildHtml(rootNode, sb);
sb.Append("</ul>");
}
}
return sb.ToString();
}
/// <summary>
/// Recursively build childNodes for a sitemap
/// </summary>
/// <param name="rootNode"></param>
/// <param name="sb"></param>
/// <returns></returns>
private static StringBuilder BuildSitemapChildHtml(SiteMapNode rootNode, StringBuilder sb)
{
foreach (SiteMapNode node in rootNode.ChildNodes)
{
if (node.HasChildNodes)
{
sb.AppendFormat("<h3><a href='{0}' runat='server'>{1}</a></h3>", node.Url, node.Title);
sb.Append("<ol>");
foreach (SiteMapNode child in node.ChildNodes)
{
if (child.HasChildNodes)
{
sb.AppendFormat("<h3><a href='{0}' runat='server'>{1}</a></h3>", child.Url, child.Title);
sb = BuildSitemapChildHtml(child, sb);
}
sb.AppendFormat("<li><a href='{0}' runat='server'>{1}</a></li>", child.Url, child.Title);
}
sb.Append("</ol>");
}
else if (rootNode == SiteMap.RootNode)
{
sb.AppendFormat("<h3><a href='{0}' runat='server'>{1}</a></h3>", node.Url, node.Title);
}
else
{
sb.AppendFormat("<li><a href='{0}' runat='server'>{1}</a></li>", node.Url, node.Title);
}
}
return sb;
}
#endregion
}
}Building the Sitemap.aspx page
Now that you have your sitemap and the helper class, building the Sitemap.aspx page is extremely simple. Calling the following from a normal .aspx page will create a tree structure containing links to all of the pages in your site:
<div id="Sitemap">
<%=SevenLabs.Utilities.Web.Helper.BuildSitemapHtml(SiteMap.RootNode) %>
</div>
The surrounding DIV is just so you can style the elements to your liking.
This is the resulting .aspx page:
Helper.cs (2.61 kb)
- Posted by Matt on August 28, 2008
We own a fair amount of domains in our company, and we generally register them with our Yahoo! Small Business account to keep them all together since our first couple were registered there. While adding a few more today, we noticed that the renewal cost for domains registered at Yahoo! have gone up 251% to $34.95 (or higher!) per year after the initial registration.
Maybe this is not news to everyone because they instituted the price hike on July 1st, but we did not notice until today. It’s not too difficult to transfer your domain to another registrar but it’s certainly a pain in the butt when you have many domain names. This is a slap in the face for anyone with a domain registered with Yahoo since your are now being charged triple for the same service. This can be especially bad for the not so savvy since they may not realize that you can transfer the domains at all.
We will be migrating our 40 or so domains away from Yahoo in the near future and I’m open to suggestions for a good registrar. I am familiar with GoDaddy but I’m not particularly fond of them. If we discover an easy way to transfer domains in bulk I will share it here.
- Posted by matt on August 21, 2008
I got into a discussion with a co-worker (and PHP programmer) about the PHP language and why a new project would ever be started using it over other popular languages. He happened to be on his way out from our company and was starting at a new development company where they programmed, near exclusively, in PHP.
Disclaimer
My exposure to PHP has typically been a forced one. I never actively sought to learn the language, but I’ve had to develop in it on numerous occasions and I can get the basics done with relative ease. I don’t claim to be a PHP programmer and I would call myself a novice at best when it comes to writing code in the language. I am a C# developer by trade.
To me, PHP does not seem to be a good choice for starting new projects that you plan on being successful. Every time I’ve seen a large PHP application, it just looked like a complete mess that nobody but the most familiar with the application could comprehend. The language was not designed to be object oriented (assuming you feel that OO is a huge step forward like I do), and they are just pushing some essential functionality to the language in the upcoming release of PHP 5.
PHP has always seemed like more of a client side language to me, probably because it’s basically a scripting language. It reminds me of classic ASP which I don’t care to be reminded of.
It has the following perceived problems in my eyes:
- It’s feature set seems crippled in comparison to other languages.
- It seems to take more code to accomplish the same task as other languages.
- You need to use 3rd party frameworks to get acceptable performance and scaling, let alone to get needed functionality.
- There is no good IDE for it that I have found.
- Most modules are not thread safe.
- There are no namespaces (yet)
- Configuring PHP can be a major pain in the ass, especially on Windows IIS.
- Session handling sucks.
- Object Oriented development is difficult, even with the new features. It just seems crammed in there.
I If you were going to create any type of serious program for the web, wouldn’t it make more sense to use a more robust language like Java or a .NET language? Yet, very large, very successful web applications are built on PHP (Wikipedia, Digg, Facebook, the list goes on…). New and innovative web applications are constantly being developed in PHP. Entire web development companies code in nothing but PHP! WHY!?
Reasons for using PHP?
Is it because there is less overhead with the code? I admit that I use .NET along with its AJAX.NET additions to write applications on a regular basis, and it sure does add a lot of unneeded overhead (not to mention ViewState). But it lets me write code so much faster by having access to the added functionality when I need it.
Is it because it’s free? I’ve heard that argument as a major selling point over and over again, yet just about every PHP developer I’ve met programs on Windows or a Mac. Besides, Java is free, the .NET language is free, and so are its (express) development tools. True, to avoid headaches you need a Windows server to deploy .NET applications to, but Java will run on that same LAMP server, just toss TomCat on it.
Is it because it’s easy to learn? Maybe. I would have to say that it’s got a flatter learning curve than the other languages and may be more inviting to the newbie.
One thing is for certain, PHP development isn’t going on in the enterprise, and for good reason.
A plea for help
Can someone tell me WHY you would start a new application in PHP? What are the benefits? If you knew PHP equally as well as Java or a .NET language, why would you code in PHP?
- Posted by Matt on June 9, 2008
I had a great idea for a Zune Originals inscription. Unfortunately for me I already bought one and couldn't think of anything to write on it at the time.
- Posted by matt on June 3, 2008
I recently had a need to track referral codes for ad campaigns as they came in through Google ad clicks. This same module can easily be used to handle affiliate codes as they come in. This module will be executed at each request, will check for a query variable, and store the value in a cookie. You can then retrieve the cookie where ever you need it in you app.
The HTTP Module
/// <summary>
/// Adds a cookie to the clients machine to track what ref code they came in through
/// </summary>
public class RefCodeExtraction : IHttpModule
{
public void Dispose()
{
}
public void Init(HttpApplication context)
{
context.BeginRequest += new EventHandler(context_BeginRequest);
}
void context_BeginRequest(object sender, EventArgs e)
{
HttpContext context = (sender as HttpApplication).Context;
//replace "q" with your variable name
if (!String.IsNullOrEmpty(context.Request.QueryString["q"]))
SaveRefCodeToCookie(context.Request.QueryString["q"], context);
else//ADD A DEFAULT CODE
SaveRefCodeToCookie("ORGANIC", context);
}
private void SaveRefCodeToCookie(string refCode, HttpContext context)
{
try
{
context.Response.Cookies.Remove("RefCookieName");
HttpCookie c = new HttpCookie("RefCookieName");
c.Value = refCode.EndsWith("/") ? refCode.TrimEnd('/'):refCode;
c.Expires = DateTime.Now.AddDays(30);
context.Response.Cookies.Add(c);
}
catch {}
}
}Activate the Module
To activate this module, place the class in your App_Code directory and add the class name to the web.config file under the HttpModules section. For example:
<httpModules>
<add name="RefCodeExtraction" type="RefCodeExtraction"/>
<add name="ScriptModule" type="System.Web.Handlers.ScriptModule, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35"/>
</httpModules>It will now be executed at the beginning of each request.
Retrieve the Ref Code from the Cookie
To retrieve the code later on, just call:
string refCode = HttpContext.Current.Request.Cookies["RefCookieName"].Value;
Conclusion
How you use the referral code or ad code is up to you. In my case, the ad campaign code is attached to a google ad using the variable "q" followed by the campaign code. This is set in the ad management system and allows for our web site to track and persist which ad codes are generating conversions. This could be adapted to pay out affiliates for referrals or whatever else you might need.
The code is below.
RefCodeExtraction.cs (1.24 kb)
- Posted by matt on May 28, 2008
A very common requirement for web applications is that they be configurable in an easy way. A lot of times it doesn't make sense to use a database to store some simple settings for your web site, especially if the application doesn't require one in the first place. There are a ton of ways to accomplish this task, but it seems to me that XML is perfectly suited for this problem. I don't mean to presume that it is better than other solutions but in case it fits the bill for your project, here is once way it can be used.
In this post I'll demonstrate one simple way to use XML to retrieve, edit, and save settings for your ASP.NET application.
The Scenario
You're building an ASP.NET application in C# that needs to have some user configurable settings. You don't want to use a database to store these settings and you want the ability to edit and save the settings through a web page. The edit page should be created dynamically from the settings contained in the XML file so that the page will not have to be updated if a setting is added.
Limitations of this example
To begin, this is not a "killer" solution to settings management. You can find a much more capable approach to XML settings by viewing the source of BlogEngine.NET and its setting management system.
This example will not allow for a user to add settings on their own that will be automatically recognized by the code. If a new setting is to be added, it will need to have a property set up and defined in the Settings class as well as be added to the Settings.xml file.
This example will not cover the vast possibilities for sub-settings you can obtain using element attributes and many other XML features.
Mostly, this is just a demonstration of how to accomplish the simple goal of storing and retrieving settings in XML.
Step 1: Setup
To begin, I would create an XML file in the App_Data directory of your application. For this example, I will call the file Settings.xml. If you're using Visual Studio just right click the App_Data folder and Add a new item. The file will be pre-filled with the standard xml encoding:
<?xml version="1.0" encoding="utf-8" ?>
You should then add a root element to the XML file. I chose the root element to be <settings></settings>. Within those tags, all of your settings will be listed out as child elements. So far the file looks like this:
<?xml version="1.0" encoding="utf-8" ?>
<settings>
</settings>
Save the file to the App_Data folder.
Step 2: Create the Settings.cs class
Next, create a Settings class in your App_Code folder (or name it whatever you like). This class will be responsible for retrieving and updating the data in the Settings.xml file. Set the class to static so that it is only loaded once for the application.
Step 3: Define the setting elements and properties
You should now begin entering the settings you'll be using for the application. In the Settings.xml file, create an element for each of the settings you will need for your program and pre-populate them with defaults if you like.
Tip: If your setting contains any markup or unsafe characters, surround the settings in the <![CDATA[ your_setting_value_here ]]> tag to avoid xml errors.
I've added some example setting elements to my Settings.xml file below:
<?xml version="1.0" encoding="utf-8"?>
<settings>
<MaxImageUploadSizeKB>1000</MaxImageUploadSizeKB>
<ThumbnailWidth>100</ThumbnailWidth>
<ThumbnailHeight>90</ThumbnailHeight>
<UploadedImageDirectory>~/images/postImages/</UploadedImageDirectory>
<SessionExpiresMinutes>20</SessionExpiresMinutes>
</settings>Next, for each of the settings you added to the Settings.xml file, create a property for it in the Settings.cs class. I'm assuming >= .NET 2.0 for the code here.
public static class Settings
{
/// <summary>
/// The maximum file size in KB for uploaded images. [Default: 1000KB]
/// </summary>
public static int MaxImageUploadSizeKB { get; set; }
/// <summary>
/// The whole number integer width of the image thumbnails [Default:100]
/// </summary>
public static int ThumbnailWidth { get; set; }
/// <summary>
/// The whole number integer height of the image thumbnails [Default:90]
/// </summary>
public static int ThumbnailHeight { get; set; }
/// <summary>
/// The directory to upload and read post images from/to [Default: ~/images/postImages/]
/// </summary>
public static string PostImageDirectory { get; set; }
/// <summary>
/// The number of minutes before a checkout session expires. [Default: 20]
/// </summary>
public static int SessionExpiresMinutes { get; set; }
}Step 4: Read the XML file into the properties
I'm sure I will catch some heat for using a DataSet to read the XML file as it seems to go against the whole point of using an XML file in the first place, but I like the simplicity and readability of the code it produces...Plus it's dead easy. The following function will assign the properties in the Settings class to the values in the XML file:
/// <summary>
/// Load the settings from Settings.xml
/// </summary>
public static void LoadSettings()
{
DataSet ds = new DataSet();
ds.ReadXml(HttpContext.Current.Server.MapPath("~/App_Data/Settings.xml"));
if (ds != null && ds.Tables[0].Rows.Count > 0)
{
DataRow dr = ds.Tables[0].Rows[0];
MaxImageUploadSizeKB = Int32.Parse(dr["MaxImageUploadSizeKB"].ToString() ?? "1000");
ThumbnailWidth = Int32.Parse(dr["ThumbnailWidth"].ToString() ?? "100");
ThumbnailHeight = Int32.Parse(dr["ThumbnailHeight"].ToString() ?? "90");
PostImageDirectory = dr["UploadedImageDirectory"].ToString() ?? "~/images/postImages/";
SessionExpiresMinutes = Int32.Parse(dr["SessionExpiresMinutes"].ToString() ?? "20");
}
}Note that the ?? operator will use the value following the ?? if the expression before it is null. This allows you to hard code defaults if you desire instead of having to do so in the XML file.
Now that you have the properties filled with data from the XML file, you can use them in your application just by referencing Settings.ThumbnailWidth for example.
Step 5: Viewing / Editing the Settings file in a web page
To display the current settings in a way that will allow them to be editable, but also allow the page to work regardless of changes to the available settings, here is something you could do.
- Create a normal .aspx page for the purpose of editing the settings.
- Add a table to the page and set runat="server" and give it an ID.
- On the Page_load event, call a function to dynamically create the setting label along with a text box for changing the value.
- Provide a Save button that will write the changes back to the XML file.
The following function will accomplish the loading task (there are some css styles in this function that are not included here):
private void LoadSettings()
{
XmlDocument xDoc = new XmlDocument();
try
{
xDoc.Load(Server.MapPath("~/App_Data/Settings.xml"));
}
catch { xDoc.LoadXml("<Settings></Settings>"); }
int alt = 0;
XmlNodeReader nodeReader = new XmlNodeReader(xDoc);
while (nodeReader.Read())
{
if (nodeReader.NodeType == XmlNodeType.Element && nodeReader.Name != "settings")
{
HtmlTableRow tr = new HtmlTableRow();
HtmlTableCell c1 = new HtmlTableCell();
HtmlTableCell c2 = new HtmlTableCell();
TextBox tb = new TextBox();
c1.InnerHtml = "<strong>" + nodeReader.Name + "</strong>";
if (alt % 2 == 0)
{
c1.Attributes.Add("class", "postDetailsLabel");
c2.Attributes.Add("class", "postDetailsReview");
}
else
{
c1.Attributes.Add("class", "postDetailsLabel");
c2.Attributes.Add("class", "postDetailsReview");
}
c1.Attributes.Add("style", "padding-right:20px;");
tb.ID = nodeReader.Name;
tb.Text = nodeReader.ReadString();
tb.Width = 400;
c2.Controls.Add(tb);
tr.Controls.Add(c1);
tr.Controls.Add(c2);
settingsTable.Rows.Add(tr);
alt++;
}
}
nodeReader.Close();
}You could use the meat of that function in a number of ways to produce the html markup that fits your style. This function will produce a table with a row for each setting, a column for the setting label, and a column with a text box holding the current setting value (editable).
Step 6: Saving the setting to the XML file
Finally, the edit page should be able to save the settings back to the XML file.
private void UpdateSettings()
{
string filename = "~/App_Data/Settings.xml";
XmlWriterSettings writerSettings = new XmlWriterSettings(); ;
writerSettings.Indent = true;
//------------------------------------------------------------
// Create XML writer against file path
//------------------------------------------------------------
using (XmlWriter writer = XmlWriter.Create(Server.MapPath(filename), writerSettings))
{
writer.WriteStartElement("settings");
foreach (HtmlTableRow row in settingsTable.Rows)
{
foreach (HtmlTableCell cell in row.Cells)
{
foreach (Control ctrl in cell.Controls)
{
if (ctrl is TextBox)
{
TextBox tb = ctrl as TextBox;
writer.WriteElementString(tb.ID, tb.Text.Trim());
}
}
}
}
writer.WriteEndElement();
}
Settings.LoadSettings();
}This function iterates through each table row which should represent a setting per row and gets the value from the text box control. The value is then written to the XML file as an element value using the text box id as the element name. Finally, the LoadSettings() function is called to reload the app settings with the new values.
Conclusion
You can use this technique to store all kinds of data in XML files. I'm sure many people will disagree with this method of settings management but the concepts presented here can be used for many different types of data. The core functions in this post will probably make it back into my helper class project in some form or another.
- Posted by Matt on May 14, 2008
I realized a while ago that I was breaking the "DRY" (Don't repeat yourself) rule pretty severely when looking at my various projects as a whole. I generally focus on not repeating code within an application but I have not been careful enough with this rule when starting new projects. Many times when I need a particular function that I have written before I will just go hunt it down and copy the code into my new project. It has recently become a problem because I now have the same function littered throughout my different programs with various modifications made to each!
It's time for me to build a utility library that will become the main source for commonly used functions in my applications. The main need that I saw for this library was the simple but tedious task of creating an email message and sending it along via a web contact form. Just about every .NET web site I've ever built has needed this functionality and each time I created a new MailMessage and Email class to handle the job. This was the first thing I added to my new Utility Library.
The purpose of this post is to try and get some feedback on which functions you would include in such a library and which you would leave out. The last thing I want to do is create some bloated set of code filled with methods that are rarely used. If anyone is interested in seeing how I am going about creating this library (or would like the actual library) just let me know in the comments and I'll do my best to help out.
Here is the initial structure and the classes I'm starting with:
Each class contains a few functions that I find myself using frequently. Here is the class diagram for the few classes I have so far:
Again, if anyone wants the actual code or libraries let me know.
Help!
If you were building a utility library, what would you include or exclude? Anyone? Bueller...?
EDIT
I've added the source code for the project to the post for anyone interested. Since the post I've added a couple more functions to the library.
SevenLabs.Utilities.zip (35.55 kb)
- Posted by matt on May 9, 2008
The built in function for creating thumbnails in ASP.NET is extremely convenient and very simple to implement.
int width = 190;
int height = 190;
Bitmap source = new Bitmap("c:\someimage.gif");
System.Drawing.Image thumb = source.GetThumbnailImage(width,height,null,IntPtr.Zero); 
(31.7k)
The trouble is that it produces relatively poor quality results and excessively large file sizes. The thumbnails tend to look very muddy when using this route, but many times it's good enough for whatever your needs may be.
An Alternative
The alternative that I use regularly involves redrawing the image using the System.Drawing.Graphics library. It is very simple to implement but produces superior results if for no other reason than its file size. The following is the standard function I use for creating thumbnails.
public static Bitmap CreateThumbnail(Bitmap source, int thumbWi, int thumbHi, bool maintainAspect)
{
// return the source image if it's smaller than the designated thumbnail
if (source.Width < thumbWi && source.Height < thumbHi) return source;
System.Drawing.Bitmap ret = null;
try
{
int wi, hi;
wi = thumbWi;
hi = thumbHi;
if (maintainAspect)
{
// maintain the aspect ratio despite the thumbnail size parameters
if (source.Width > source.Height)
{
wi = thumbWi;
hi = (int)(source.Height * ((decimal)thumbWi / source.Width));
}
else
{
hi = thumbHi;
wi = (int)(source.Width * ((decimal)thumbHi / source.Height));
}
}
// original code that creates lousy thumbnails
// System.Drawing.Image ret = source.GetThumbnailImage(wi,hi,null,IntPtr.Zero);
ret = new Bitmap(wi, hi);
using (Graphics g = Graphics.FromImage(ret))
{
g.InterpolationMode = System.Drawing.Drawing2D.InterpolationMode.HighQualityBicubic;
g.FillRectangle(Brushes.White, 0, 0, wi, hi);
g.DrawImage(source, 0, 0, wi, hi);
}
}
catch
{
ret = null;
}
return ret;
}
(10.5k)
This function is handy because it's parameters include a flag for maintaining the aspect ratio of the image along with the thumbnail size you would like. The thumbnail magic happens in this portion of the code:
using (Graphics g = Graphics.FromImage(ret))
{
g.InterpolationMode = System.Drawing.Drawing2D.InterpolationMode.HighQualityBicubic;
g.FillRectangle(Brushes.White, 0, 0, wi, hi);
g.DrawImage(source, 0, 0, wi, hi);
}This method is slightly slower but the results are hard to ignore as illustrated by the comparison below:
(31.7k) (10.5k)
Pretty nice improvement in both file size and quality I would say, but....
We can do even better
Now we can add into the mix some JPEG compression and really optimize the results. I won't pretend to fully understand how the JPEG compression code below works, but it sure does the trick.
//Configure JPEG Compression Engine
System.Drawing.Imaging.EncoderParameters encoderParams = new System.Drawing.Imaging.EncoderParameters();
long[] quality = new long[1];
quality[0] = 75;
System.Drawing.Imaging.EncoderParameter encoderParam = new System.Drawing.Imaging.EncoderParameter(System.Drawing.Imaging.Encoder.Quality, quality);
encoderParams.Param[0] = encoderParam;
System.Drawing.Imaging.ImageCodecInfo[] arrayICI = System.Drawing.Imaging.ImageCodecInfo.GetImageEncoders();
System.Drawing.Imaging.ImageCodecInfo jpegICI = null;
for (int x = 0; x < arrayICI.Length; x++)
{
if (arrayICI[x].FormatDescription.Equals("JPEG"))
{
jpegICI = arrayICI[x];
break;
}
}This code will set up the encoderParameters needed for saving the new compressed thumbnail. The quality[0] value is where you set the compression level. I've had success going as low as a value of 40 for some applications, but when quality is a major requirement I find 75 to do very well. To use this engine you would execute the JPEG Compression code before you save your thumbnail, then use its encoderParamaters as a parameter when saving. For example:
System.Drawing.Image myThumbnail = CreateThumbnail(myBitmap,Width,Height,false);
//Configure JPEG Compression Engine
System.Drawing.Imaging.EncoderParameters encoderParams = new System.Drawing.Imaging.EncoderParameters();
long[] quality = new long[1];
quality[0] = 75;
System.Drawing.Imaging.EncoderParameter encoderParam = new System.Drawing.Imaging.EncoderParameter(System.Drawing.Imaging.Encoder.Quality, quality);
encoderParams.Param[0] = encoderParam;
System.Drawing.Imaging.ImageCodecInfo[] arrayICI = System.Drawing.Imaging.ImageCodecInfo.GetImageEncoders();
System.Drawing.Imaging.ImageCodecInfo jpegICI = null;
for (int x = 0; x < arrayICI.Length; x++)
{
if (arrayICI[x].FormatDescription.Equals("JPEG"))
{
jpegICI = arrayICI[x];
break;
}
}
myThumbnail.Save(Path.Combine(SavePathThumb, fileName), jpegICI, encoderParams);
myThumbnail.Dispose();
(2.39k)
Which still looks pretty darn good for 2.39k.
Conclusion and final comparison
Here is the final comparison between the 3 thumbnails going from largest file size to smallest:
Largest = 31.7k
Uncompressed redraw = 10.5k (67% smaller)
Compressed redraw = 2.39k (92% smaller)
It's hard to ignore those results. The source code for the thumbnail function and the JPEG compression engine are below.
ThumbnailGenerator.cs (1.97 kb)
JPEGCompressionConfig.cs (969.00 bytes)
- Posted by Matt on April 15, 2008
Optimizing a database can be one of the most difficult things to do. There are so many factors that come in to play that it's hard to gauge your results and identify the actual bottleneck. From network and hardware issues to poorly configured server settings, things can get frustrating really fast. There are far too many variables to consider as it is without having to worry about poorly written SQL statements.
Fortunately, inefficient SQL is the easiest performance problem to solve and typically produces the most dramatic results in an application that rely's heavily on a database. There are plenty of ways to write bad SQL but today I want to focus on an issue I come across regularly with colleagues, even some of the more experienced ones.
The dreaded subquery. Boy can they be misused and expensive. Don't get me wrong, a subquery is an extremely powerful feature of modern databases but more often than not (in my experience) it is used improperly. There are some situations when a subquery is the only answer to the problem and when that is the case I'm real pumped that it exists.
What's a subquery?
A subquery is a select statement within another select statement. For example:
Select * from orders where orders.customerID IN (Select ID from customers where customers.state = 'NY')
The statement inside the ( ) will execute separately from the main query and it's results will be held in memory to be used for the original query. This query will give you all orders whose customers state is NY.
What's bad about subqueries?
Not a lot on their own, but if you're not careful they can cause serious performance problems. Depending on your goal you can quickly get into several nested sub query statements, each of which rely on the furthest inner query to complete first. If any of the queries are poorly written, say missing a limiting WHERE condition, the entire chain collapses. Subqueries do not take advantage of the database engine properly either and typically produce much slower results.
Have no fear, the GROUP BY clause is here!
In the majority of the situations that I have seen, a sub query could be replaced with a JOIN statement or with a GROUP BY clause. I don't want to get into a long discussion about how JOIN operations work but I thought it would be helpful to show an example GROUP BY statement. Most times when a sub query is used it's only because the writer is unaware of the group by clause or how to use it.
Suppose you have a database table holding a bunch of orders and you have a STATE column and an ORDER_AMOUNT column.
You want to know the total order amount per state in 2007.
This could be accomplished using a subquery in the following way:
select STATE as oState, (SELECT SUM(ORDER_AMOUNT) from OrderData
where RECEIVED_DATE between '20070101' and '20080101' and STATE = oState) as Dollars
from OrderData where RECEIVED_DATE between '20070101' and '20080101' order by oState asc
That code will work, but the query will be inefficient. Running this query on an actual set of data (not the identical query) with around 145,000 rows took 13.09 seconds in MySql according to the query browser.
This query can be re-written using the GROUP BY clause which will take advantage of the database engine to produce the same results. The same query can be written as follows:
SELECT STATE, SUM(ORDER_AMOUNT) as DOLLARS FROM OrderData
where RECEIVED_DATE between '20070101' and '20080101' GROUP BY STATE HAVING SUM(ORDER_AMOUNT) > 0;
In comparison, this query which produced the same result against the same set of data took .07 seconds!
That is 187 times faster. More than a little bit.
Obviously this is a drastic result, and honestly I didn't plan that or expect it but it sure illustrates my point. It's less than scientific since you don't know my environment, the actual query, or the actual data but I did try my best to make it legit. Making that change in a production environment could save an immense amount of processing time.
What's the point?
Nothing really, I just realized that a lot of people are unaware of the group by clause and all they need is to see it once to change their ways. So if you know a database beginner or a database "expert" that could use a wake up call, pass them this article and help them out.
- Posted by matt on April 8, 2008
I've been using the excellent .NET blogging platform BlogEngine.NET for a while now and really enjoy coding against it even though code highliting doesn't seem to work. The extensibility of the code is awesome and allows you to morph the program into whatever you like. I was recently converting the program into a CMS system for a project that required a simple management tool and realized that the Page listing was lacking a bit in its display. Granted, this tool was not meant to be a CMS system but if your blog has many pages this might be useful to you as well. The system has built in Parent -> Child paging but displays all of the pages straight out in a list. I wanted to see the list with the Parent pages in bold and the child pages indented beneith its parent. To accomplish this I added 2 properties to the Page.cs object in the Blogengin.core, then I added a function to the Pages.cs code behind in /Admin/Pages/Pages.aspx.cs and modified the BindPageList() function to call it.
Note: I recompiled the program as a .NET 3.5 project to make use of Lambda expressions in the BindChildPageList() function. Change it to use delegates in the for loop if you're using .NET 2.0
First I added two properties to the Page object to make things easier later on: [BlogEngine.Core/Page.cs]
///
/// Does this post have a parent page?
///
public bool HasParentPage
{
get { return this.Parent != Guid.Empty; }
}
///
/// Does this post have child pages
///
public bool HasChildPages
{
get
{
foreach (Page p in Page._Pages)
{
if (p.Parent == this.Id)
return true;
}
return false;
}
}Then I modified the BindPageList() function [BlogEngine.Web/Admin/Pages/Pages.aspx.cs]
private void BindPageList() {
foreach (Page page in BlogEngine.Core.Page.Pages) {
if (!page.HasParentPage)//MOD: Only add the page to the list if it's a root page (children are added below)
{//MOD
HtmlGenericControl li = new HtmlGenericControl("li");
HtmlAnchor a = new HtmlAnchor();
a.HRef = "?id=" + page.Id.ToString();
a.InnerHtml = page.Title;
System.Web.UI.LiteralControl text = new System.Web.UI.LiteralControl
(" (" + page.DateCreated.ToString("yyyy-dd-MM HH:mm") + ")");
li.Controls.Add(a);
li.Controls.Add(text);
if (page.HasChildPages)//MOD: If the page has children, add them recursively
{//MOD
//find children
li.Controls.Add(BuildChildPageList(page));//MOD
}//MOD
ulPages.Controls.Add(li);
}//MOD
}
divPages.Visible = true;
aPages.InnerHtml = BlogEngine.Core.Page.Pages.Count + " " + Resources.labels.pages;
}
And finally I added the BuildChildPageList() function [BlogEngine.Web/Admin/Pages/Pages.aspx.cs]
private HtmlGenericControl BuildChildPageList(BlogEngine.Core.Page page)
{
HtmlGenericControl ul = new HtmlGenericControl("ul");
foreach (Page cPage in BlogEngine.Core.Page.Pages.FindAll(p => (p.Parent == page.Id)))
{
HtmlGenericControl cLi = new HtmlGenericControl("li");
cLi.Attributes.CssStyle.Add("font-weight", "normal");
HtmlAnchor cA = new HtmlAnchor();
cA.HRef = "?id=" + cPage.Id.ToString();
cA.InnerHtml = cPage.Title;
System.Web.UI.LiteralControl cText = new System.Web.UI.LiteralControl
(" (" + cPage.DateCreated.ToString("yyyy-dd-MM HH:mm") + ")");
cLi.Controls.Add(cA);
cLi.Controls.Add(cText);
if (cPage.HasChildPages)
{
cLi.Attributes.CssStyle.Remove("font-weight");
cLi.Attributes.CssStyle.Add("font-weight", "bold");
cLi.Controls.Add(BuildChildPageList(cPage));
}
ul.Controls.Add(cLi);
}
return ul;
}
The result is the following:
