ReBuildAll Blog
Thoughts (mostly) on .NET development

Plug & Play MVC   (MVC)   
This entry will be available in Finnish only - it is related to the TechDays 2011 speech on 31.3 about extending MVC framework.

Pidin esityksen TechDays 2011 MVC kehikon laajennuksesta. Esityksessä käsiteltiin miten MVC kehikkoa voi laajentaa eri paikoilla.

Itse kehikko on iso kokoelma pieniä osia mitkä toimivat hyvin yhdessä. Kuitenkin näitä pieniä osia voidaan muokata ja räätälöidä, tai jopa tehdä omia osia mitkä käytetään alkuperäisten sijasta. Näin voimme räätälöidä itse kehikonkin että se toimii juuri niin kuin me haluamme.

Näitä laajennuspisteitä on monta, ja esitys yrittää antaa kartan näihin pisteisiin. Ohjeiden perusteella pitäisi olisi helppo löytää juuri se laajennuskohta, mikä antaa mahdollisimman paljon hyötyä.

Esityksen PDF kalvot ovat ladattavissa täältä.

Esityksen demo sovellus on myös ladattava täältä.

Demo vaatii Visual Studio 2010 version (Express pitäisi toimia myös) sekä SQL Server Express version. Kun sovellusta käynnistää ensimmäinen kertaa, se luo itselleen tietokannan (App_Data kansioon). Kuitenkin pitää käsin ajaa alustus. Se löytyy /Home/Setup osoitteesta sovelluksen alta (eli HomeController:in Setup action).

jQuery validate and the comma decimal separator   (MVC)   
In response to the comments I have received to the original article, I have updated it with information how to modify the behavior of the jQuery validator plugin without directly editing the source.

While working with MVC 3, I hit some problems with validation. MVC 3 uses jQuery validation plugin with unobtrusive validation to help you out. Except when I had to use Finnish formatted numbers, there was a problem. jQuery/javascript did not really like the comma as a decimal separator.

A similar problem is with the range validation, which also refuses to work when using a comma as the decimal separator.

A rather nasty set of problems I might say, especially since number validation rules are added AUTOMATICALLY by ASP.NET MVC if you render a TextBox for a Decimal field in the model. It just seems the writer of the validator plugin forgot about everyone outside US (thanks @Deef for this last comment).

Fixing jquery.validate.js

You can of course go into the source of the validator javascript file and make your changes there, but modifying this file should never be done directly. That would create all kinds of nasty problems when updating to future versions.

Instead, one should take advantage of extending (replacing) the validator functions that are provided by jQuery Validate. (thanks to @ruben for the information on this).

To fix these problems, we can take the default implementation from the jquery.validate.js file for the range() and number() functions. We then create another .js file (say jQueryFixes.js), in which we override these default functions with ones that contain support for the comma as a decimal separator. The contents of the file should be something like this:

$.validator.methods.range = function (value, element, param) {
    var globalizedValue = value.replace(",", ".");
    return this.optional(element) || (globalizedValue >= param[0] && globalizedValue <= param[1]);
}

$.validator.methods.number = function (value, element) {
    return this.optional(element) || /^-?(?:\d+|\d{1,3}(?:[\s\.,]\d{3})+)(?:[\.,]\d+)?$/.test(value);
}


This should be the last .js file you include, so it overrides the behavior of the default functions from the jQuery Validator plugin.

For number validation, what we have done is replace the last dot with an expression that also accepts a comma (change is bolded):

return this.optional(element) || /^-?(?:\d+|\d{1,3}(?:,\d{3})+)(?:[\.,]\d+)?$/.test(value);

This will solve the number validation problem with the decimal point/comma. If you are dealing with thousand separator, you might need to tweak this further.

As for the range validation, I opted to simply replace the comma with a dot, and proceed normally from there on. You could of course use the jquery globalization plugin, please see the comments section for @ruben's recommendation on that one.

And that is about it. Now decimal separator commas should also be accepted by the validators.

Serve MVC Views from a ZIP file   (MVC)   
Some time ago I wrote an article about MVC and virtual views. I made a comment that you could create a VirtualPathProvider that would serve content from a ZIP file. Some of you commented you would like to see an implementation for that. Well, here it is.

Please read that article for more information on how to implement a VirtualPathProvider. Here I will not go into details on everything involved.

You can download the entire example solution (Visual Studio 2010 and MVC 3 required!) from here.

The example should run as soon as you load it up in VS2010. Try to request the following addresses (relative to the application root): /Test and Test/Complex for non zipped versions of views. And /TestZipped and /TestZipped/Complex for zipped versions of the views.


How to extract files from a .ZIP file?

For this example, I will use SharpZipLib, which a freely available .zip file management library. It is already included in the example solution I linked at the beginning of this article. So if you downloaded that you will not need to download SharpZipLib separately.


Preparing to serve Views from a .ZIP file

I decided that the content I want to serve will be stored in a file called Views.zip, that needs to be stored at the root of the application (~/Views.zip). This .zip file has to have the same directory structure as the real MVC application. So we need a Views folder, and inside that, the controller folder. Inside the controller folder (or folders) there will be the actual views.

For the example solution I put the views for the "TestZipped" controller into the Views.zip file. There are two views, Index.cshtml and Complex.cshtml. Both include Razor markup to demonstrate that the solution demonstrated is actually fully functional. You could of course pack up the entire Views folder in the example solution, and it will still work :)

(I did try this, and it did work. If you do it, make sure you still leave web.config in place, as that is outside the scope of VirtualPathProvider. But you can just put everything else in the Views/ folder into the .ZIP file, and delete them from the actual folder. The application will still work! :))

The directory structure inside the .zip will make it easy to find and extract the files. Because the VirtualPathManager receives virtual file names, like /Views/TestZipped/Complex.cshtml, when we have the same structure in the .zip, it is easy to string compare the names.

Of course you could implement as complex a logic as you want to find the proper file :)


Creating a VirtualPathProvider for serving content from .ZIP files

The next step is to create a new VirtualPathProvider. Please note that the implementation I provide is for demonstration purposes only, and might not be fully ready for production use.

    public class ZippedVirtualPathProvider : VirtualPathProvider
    {
        public static void RegisterMe()
        {
            HostingEnvironment.RegisterVirtualPathProvider(new ZippedVirtualPathProvider());
        }

I also created a little helper method to register our path provider.

We will override FileExists(), GetFile() and GetCacheDependency(). Let's start with FileExists().

We simply want to check if we have the .zip file is available and if the file can be found in the .zip file. If both conditions are true, we can indicate we found the file. Otherwise, we let the default implementation take over and perform any checks needed.

        public override bool FileExists(string virtualPath)
        {
            if (IsViewsZipFound && FileExistsInZip(virtualPath))
            {
                return true;
            }
            else
            {
                return base.FileExists(virtualPath);
            }
        }

        private bool IsViewsZipFound
        {
            get
            {
                if (File.Exists(ViewsZipPath))
                {
                    return true;
                }
                else
                {
                    return false;
                }
            }
        }

        private static string ViewsZipPath
        {
            get
            {
                return HttpContext.Current.Server.MapPath ( "~/Views.zip" );
            }
        }

        private static string TransformVirtualPath2ZipPath(string virtualPath)
        {
            string fullPath = virtualPath;
            if (fullPath.StartsWith("~/"))
            {
                fullPath = fullPath.Substring(2);
            }
            if (fullPath.StartsWith("/"))
            {
                fullPath = fullPath.Substring(1);
            }
            return fullPath;
        }

        private bool FileExistsInZip(string virtualPath)
        {
            string zipPath = TransformVirtualPath2ZipPath(virtualPath);

            using (ZipInputStream s = new ZipInputStream(File.OpenRead(ViewsZipPath)))
            {
    			ZipEntry theEntry;
                while ((theEntry = s.GetNextEntry()) != null)
                {
                    if (theEntry.IsFile && theEntry.Name == zipPath)
                    {
                        return true;
                    }
                }
            }

            return false;
        }

So to check if the .zip exists, we just use the System.IO.File.Exists() method. To verify if the file exists, we read the .zip file entries and try to find one that matches our file. If we find it, we return true. Simple enough.

For getting the file, we use similar code:

        public override VirtualFile GetFile(string virtualPath)
        {
            if (IsViewsZipFound)
            {
                var file = ExtractFromZip(virtualPath);
                if (file != null)
                {
                    return file;
                }
            }

            return base.GetFile(virtualPath);
        }

        private VirtualFile ExtractFromZip (string virtualPath)
        {
            string zipPath = TransformVirtualPath2ZipPath(virtualPath);

            using (ZipInputStream s = new ZipInputStream(File.OpenRead(ViewsZipPath)))
            {
                ZipEntry theEntry;
                while ((theEntry = s.GetNextEntry()) != null)
                {
                    if (theEntry.IsFile && theEntry.Name == zipPath)
                    {
                        byte[] data = new byte[theEntry.Size];
                        s.Read(data, 0, data.Length);
                        return new ZippedVirtualFile(virtualPath, data);
                    }
                }
            }

            return null;
        }

Here we check if the .zip file exists, and then try to extract the file from it. If it is found, it is also returned. A new class called ZippedVirtualFile is used to return the file, this will be introduced a little later below.

Finally, we override the GetCacheDependency() method, because MVC uses that to monitor when it needs to rebuild/reread a file. What we will do is instead of returning a cache dependency on the file, we return a dependency on the .zip file. If the .zip file changes we will assume our file changes. Not very robust, but will work for this example.

        public override System.Web.Caching.CacheDependency GetCacheDependency(string virtualPath, System.Collections.IEnumerable virtualPathDependencies, DateTime utcStart)
        {
            if (IsViewsZipFound && FileExistsInZip(virtualPath))
            {
                return new System.Web.Caching.CacheDependency(ViewsZipPath, utcStart); 
            }
            else
            {
                return base.GetCacheDependency(virtualPath, virtualPathDependencies, utcStart);
            }
        }


ZippedVirtualFile

We need a little helper class that will hold our extracted file, and return a stream to it on demand.

    public class ZippedVirtualFile : VirtualFile
    {
        public byte[] data;

        public ZippedVirtualFile(string virtualPath, byte[] data)
            : base(virtualPath)
        {
            this.data = data;
        }

        public override System.IO.Stream Open()
        {
            return new MemoryStream(data);
        }
    }

The class is very simple, it simply holds a byte array, and when a stream is requested, returns a new MemoryStream that can be used to read the contents.


Using the ZippedVirtualPathProvider

As a final step, we need to register our provider. This can be done in global.asax.cs by adding the following code to the Application_Start() method:

        protected void Application_Start()
        {
            // ...

            ZippedVirtualPathProvider.RegisterMe();
        }


And we are ready to go! (As soon as we create the Views.zip file of course).



SQL Server User Instances and the Database 'abc' already exists mystery   (Tips & Tricks)   
I was trying to find a solution to a problem with LINQ and its CreateDatabase() call (see details in another blog post). Then I bumped into the following problem.

When I tried attaching a database into an SQL Server Express User Instance that existed before (the same database under the same path), I received an exception. I actually manually deleted the .MDF and .LDF files from disc, and then tried to recreate them programatically. The exception informed me the database file already existed. Although I have deleted it myself.

The exception said: Database 'path_to_database' already exists. Choose a different database name.. Which was very strange.

Reading more about SQL Server Express User Instances, I found out that they are more complicated than I thought. What actually happens is that SQL Server Express copies the master and msdb databases under the user's directory (who runs the user instance). It then starts the user instance (sqlservr.exe) under the user account. When you want to use a database in a user instance, it actually attaches the database file to the user instance - just as you would attach a database to a regular SQL Server instance. It just happens behind the scenes.

This creates trouble of course, because the database is registered somewhere, and you shouldn't just delete the files. Which I did. And that is why I got the error message. :)

There is some information about a tool that you can use to connect to a user instance and execute commands (detach maybe?). I found it too complicated. (But if you are interested, visit this MSDN page for more information!)

Rather, I killed my user instance process (Task Manager, kill sqlservr.exe). Then proceeded to delete my user instance directory. For Windows 7, this would be:

c:\Users\username\AppData\Local\Microsoft\Microsoft SQL Server Data\SQLEXPRESS\

I then restarted my application (which called CreateDatabase()), and it worked. It recreated the above directory, and everything was fine.

I have to admit that this seems like a bit of a drastic solution, but it sure is faster than connecting to the instance and issuing SQL commands by hand :)

And it should not have any side effects, because all user instance databases will just get attached automatically the next time you use them.

LINQ CreateDatabase cannot resolve DataDirectory   (ASP.NET)   
You might be familiar with SQL Server Express User Instances. These are per user copies of the database server, into which you can attach files "on the fly", without a real need for a fully managed SQL environment. You probably bumped into it if you have used the default setup for MVC or ASP.NET where you tried to use membership or other services. ASP.NET created a .MDF and .LDF file for you in the App_Data directory automatically. These are actually SQL Server Express User Instance databases.

If you ever checked the ConnectionString, it will contain a |DataDirectory| directive (and the term User intance=true. At runtime, the ASP.NET runtime resolves this to point into the App_Data folder of the application. The SQL client runtime of .NET is able to connect up this file into the user instance and use it, without any user action needed. It can also create it, if it does not exist.

While ASP.NET provides this behavior for its own databases by default, there is nothing stopping us from using the same feature with our custom application databases.


Create a missing database

When I put code into source code control, I rarely put it actual .MDF and .LDF files (databases). Instead, usually an .SQL script goes in there that can create my database. However, since LINQ has the neat feature to create the missing database, why not use that? This way I could make my project create the database it needs by itself, when starting up.

It would be ideal for distributing demos and examples. The person who opens it up, just needs to start it. Visual Studio self hosts the web application in its own web server, the database is created in the SQL Server Express User Instance, and it just starts working.

It is all very simple in theory, until you want to call LINQ's CreateDatabase() on a LINQ model that has a ConnectionString with the |DataDirectory| variable. I used CreateDatabase() succesfully before with regular SQL server databases. But when I tried the user instance approached, it failed.

My first try was the following code:

MyDataContext ctx = new MyDataContext( connString )
if (ctx.DatabaseExists() == false)
{
    ctx.CreateDatabase();
}


And what I got was an ArgumentException, that told me: Illegal characters in path.. Strange. I checked the callstack, which was:

   at System.IO.Path.CheckInvalidPathChars(String path)
   at System.IO.Path.NormalizePath(String path, Boolean fullCheck, Int32 maxPathLength)
   at System.IO.Path.GetFullPathInternal(String path)
   at System.IO.Path.GetFullPath(String path)
   at System.Data.Linq.SqlClient.SqlProvider.System.Data.Linq.Provider.IProvider.CreateDatabase()
   at System.Data.Linq.DataContext.CreateDatabase()


I checked everything, but all seemed to be fine.

I then tried substituting the actual path in the connection string, and my code worked. So what is going on?


Bug in LINQ DataContext

LINQ DataContext uses its own SQL wrapper, and that wrapper has a bug. It cannot resolve the |DataDirectory| constant. I used to .NET Reflector to verify this. Sure enough, the System.Data.Linq.SqlClient.SqlProvider class retrieves the AttachDbFilename entry from the connection string, and then uses it directly. So if you have |DataDirectory| in there, you are out of luck: an exception will inform you of this :)

Luckily, you can retrieve the value of the DataDirectory entry from the current AppDomain. So you could do the substitution yourself, pass the corrected connection string to the DataContext and create the database. Voila!


DataDirectory

So first, we need the DataDirectory entry. We can use the following code to retrieve it from the current AppDomain:

string dataDirectory = AppDomain.CurrentDomain.GetData("DataDirectory").ToString();


Putting it together

So I ended up writing the following method into my Global.asax.cs file:


        private void EnsureDatabase()
        {
            string connString = null;
            connString = global::System.Configuration.ConfigurationManager.ConnectionStrings["MyConnectionString"].ConnectionString;

            if (connString.Contains("|DataDirectory|"))
            {
                string dataDirectory = AppDomain.CurrentDomain.GetData("DataDirectory").ToString();
                connString = connString.Replace("|DataDirectory|", dataDirectory);
            } 

            using ( MyDataContext ctx = new MyDataContext( connString ) )
            {
                if (ctx.DatabaseExists() == false)
                {
                    ctx.CreateDatabase();
                }
            }
        }


This code loads my connection string and if it finds the |DataDirectory| directive in there, it will replace it with the DataDirectory value from the AppDomain. It then fires up an instance of the DataContext and makes sure the database exists. Sure enough, this works (except ... see below).

This code can be called in Global.asax.cs in the Application_Start() method. After this, whenever you run the application and it finds the database is missing, it will create it. You would still need to populate it with initial data, if that is a requirement, because the database will be empty of course.


Multiple user instance databases

While testing the above problem, I replace the DataDirectory variable by hand first, tried it that way, and it worked. I then stopped the web server, deleted the created files and started playing around. I bumped into a problem when I then wanted to create the files again. As if SQL server believed the files existed, although I deleted them. As it turns out, User Instances are not as simple as they sound. There is a slight problem when you want to attach a database with the same path into the user instance that already existed. I wrote a separate blog post about that. See it here.