RSS

Monthly Archives: September 2011

Microsoft Sync Framework

I was excited to learn that Microsoft has developed a sync framework which is free and available for download. Programming for the web normally saves the need to sync stuff, because there are no client databases which can go out of sync, and server-side databases have their bulit-in replication solutions. So there’s usually very little development that you must do on your own for syncing data sources. There are also solutions for syncing file systems, so you may choose these when required. However, there are certain scenarios in which you have to develop sync code yourself. These include for example syncing sources from a remote hardware which is “occasionally connected”, such as mobile or tabular devices; Non-web solutions for desktops; syncing between databases which are not of the same type (Oracle and SQL Server etc.); or a mere custom action involving sync in your application. While sync in itself isn’t rocket science, it’s still a non-trivial, time-consuming and bug potential feat. That’s why I wanted to learn a little more of this sync framework, which has been around for quite some time now.

I was hoping that as the Sync framework was around for some time, that I’ll find plentiful of blog, samples and articles. I was also hoping that the official sync web site would be friendly and that it’ll get me up and running in no-time. This wasn’t the case. While there was a useful sample code for using the file synchronization, available directly from the sync home page, the database samples were no where to be found, and the home page was filled with datasheets and architecture descriptions which may be important, but lack the samples to accompany them. It took me some time to detect the samples after installing the SDK, in the Program Files folders, as there’s no shortcut from the start menu to that location. The SDK was also a pain to install because there are multiple downloads involved instead of a single file installation. In particular, it seems like MS forces you to answer a survey in order to download the v4 CTP. In order to hasten installation, I suggest that you go to the v4 CTP “installation” url, scroll down and follow the instructions according to your OS (32/64 bit). Regardless, the samples here were created in v2.1 of the sync framework and require the following installations:

  1. SyncSDK v2.1.
  2. Synchronization v2.1.
  3. Database Providers v3.1.

If you’d still like to download the v4 CTP, go here. According to the documentation, the November Refresh contains an iPhone sample and a “Tooling Wizard UI” which allows you to configure sync related operations.

File sync

I decided to write about File sync first, as this seemed most trivial and easy to accomplish using MS’ built-in file sync provider and sample. In order to implement a file sync solution, simply add references to Microsoft.Synchronization and Microsoft.Synchronization.Files assemblies from your project, create dummy folders for source and target directories with dummy files, and write the following code:

That’s it! Several lines of code and we have a fully operational file sync, including sub folders sync. I find it quite amazing, and it can give us some idea as to how easy it should be to sync other providers. The code is self explanatory, but I do have some comments:

  1. The default sync direction is bi-directional Upload and Download, but this can be controlled via the SyncOrchestrator.Direction property.
  2. The FileSyncProvider different constructors work with “local file paths”, but it also works with a UNC path. It does not seem to be working with “file://” and I assume that it does not work with “ftp://” paths either.
  3. The FileSyncProvider creates a metadata file, which contains replica information required for the synchronization.
  4. You can provide a FileSyncScopeFilter file filter object to the FileSyncProvider constructors, thus have more control over which files are synced.
  5. You can set the options enumeration flags to recycle the different overwritten files instead of permanently deleting them.
  6. The Synchronization( ) method returns a stats object containing basic data about the completed synchronization.

The above sample is the most basic a developer may require for synchronization. On top of this sample, you can bind synchronization events and conflict resolution. For example, you can wire an event handler to the ApplyingChange event and control whether files should be skipped synchronization at runtime. Note that this will only work if FileSyncProvider.DetectChanges( ) is called either implicitly by the SyncOrchestrator.Synchronize( ) method, or explicitly if the FileSyncProvider options specifies that DetectChanges is to be called explicitly.

Note: If you come across the following exception, check out that the project file is set to compile to the correct platform or to “Any CPU”:

Retrieving the COM class factory for component with CLSID {031913FE-EB2A-49F2-B2BE-B996B2448CD4} failed due to the following error: 80040154 Class not registered (Exception from HRESULT: 0x80040154 (REGDB_E_CLASSNOTREG)).

SQL Server sync

Now I felt like it was time to move on to what I considered the major requirement for the sync framework – database synchronization. I decided to settle for synching 2 SQLServer Express schemas at first. No Compact, Azure or another non-SQL DB – just to get things started. I setup two SQL Server 2008 Express databases and wrote the following code, which was simply based on the FileSyncProvider written earlier:

I really didn’t know what to expect, because I could not find a simple straight forward example. I read that the sync creates it’s own metadata database independently from the actual data being synchronized, but I didn’t know if that will be automatic like the FileSyncProvider had done. So, at first, I got the following error:

Retrieving the COM class factory for component with CLSID {EC413D66-6221-4EBB-AC55-4900FB321011} failed

Apparently you have to install the Synchronization-v2.1-x86-ENU.msi in order to use database synchronization from VS2010, even if you’re running on a x64 based machine. I installed, ran the code and this time I got the following DbNotProvisionedException error:

The current operation could not be completed because the database is not provisioned for sync or you not have permissions to the sync configuration tables.

Unlike FileSyncProvider, this isn’t entirely an automated process. You have to “provision” your databases first in order to get this to work. In the Program Files Sync SDK folder, there are some examples which help out to understand how to do this, specifically SharingAppDemo-CEProviderEndToEnd and WebSharingAppDemo-CEProviderEndToEnd projects in the SDK folder. After experimenting a little, I came up with the following code:

Note lines 22-23. Line 22 prepares the server for provisioning, and line 23 prepares the client for provisioning. In the sample code from the SDK, comments which accompany the server side provisioning samples indicate that this code is typically run at “design time”, and not at sync time, and that it should not be “baked” into the client application. So bear in mind that in real-world applications, lines 22-23 should not co-exist next to each other.

Also note lines 18-19. As you can see, there’s a provisioning “scope” which I believe allows you to synchronize specific parts of the databases, rather then synchronize entire databases.

The code for those methods is as follows:

Lines 33-47 is the method implementation for server side provisioning:

  • Line 37 checks whether provisioning is required on the server side for the given scope.
  • Lines 39-40 defines the table(s) which are included in the synchronization process for the specified scope.
  • Line 42 populates the information from the scope definition.
  • Line 44 defines the behavior upon provisioning.
  • Line 45 performs the actual provisioning.

Lines 49-58 is the method implementation for client side provisioning:

  • Line 52 checks whether provisioning is required in the client side, for the specified scope.
  • Line 54 retrieves the provisioning information from the server side (no need to redefine the tables as we did in the server side provisioning).
  • Line 55 populates the provisioning information according to the retrieved server description.
  • Line 56 performs the actual provisioning in the client.

This newer code seems to do the trick. The databases seem to be synchronizing well, post insert, delete and update statements, both ways.

I tried to rename a table column but the synchronize can’t seem to detect that change and I received an exception. I found this article, which addresses sync scope schema changes. The article states that this is one of the most requested features of the sync framework, so maybe we can expect some automatic implementation in future versions to come.

More Resources

Summary

This wasn’t as trivial as I expected it to be. The FileSyncProvider is indeed very simple to use and code, mostly because the Introduction to Microsoft Sync Framework File Synchronization Provider article is available directly from the sync home page. However, the database synchronization counterpart official document contains no sample code and focuses mainly on architecture, with no links to actual concrete code samples. I spent quite some time in detecting those code samples in the SDK (as there’s no link to them from the Start menu either). Don’t get me wrong – I think that the sync framework can be a great solution; more over, I reckon that there are official links and blogs that show more and more examples, but these should be available directly from the homepage, so developers can get started within 30 minutes to sync their databases.

BTW: Sync framework v4 CTP is from October 2010, “Refreshed” in November. That’s from last year. Is this going to be another one of MS’ deprecated projects? Except for the recently released (Sep 2011) Sync Framework Toolkit (available from the blogs), there are hardly any recent blogs. Prior to the Sync Framework Toolkit blog entry, the next recent post is dated six months ago. How do we developers know if we should put more effort and rely on this framework?

 
15 Comments

Posted by on 27/09/2011 in Software Development

 

Tags:

ASP.NET Ajax file upload using jQuery File Upload plugin

Async Ajax file upload sounded like “science fiction” to me up until some time ago. I remember those days, back in Classic ASP, when I had to employ 3rd party components to have the browser upload files. When ASP.NET introduced FileUpload control, I was truly excited. It was plain easy to use. However, it still required a full postback. Several months ago I saw that AjaxControlToolkit provided an AsyncFileUpload control, and I think that this was the first time I’ve heard of the idea to provide async file upload. However, a quick test I performed back then failed, and I decided to forgo the idea till bugs are fixed and things become more stable. I even developed a File Upload control in Silverlight which provided me with an async upload (as well as other features I needed back then). Basically I continued to use ASP.NET’s excellent FileUpload where possible, up until today, as I had a situation in which I preferred to check on async upload solutions again.

So, a quick search in Google led me to check out this jQuery File Uploader. It looked good, looked like it was working ok, and there was a sample code in C# (WebForms) which got me going quickly (MVC sample is also available). One last thing: although it seems to have jQuery UI support for the “look and feel”, I wanted the most basic functionality of uploading a file asynchronously as I had to reuse graphics from the existing project.

The sample code taught me that html now allows multiple file selection. This was new to me. I tried to track this down and found out the the “multiple” attribute is still in draft, but is already working in some of the newer browsers (unfortunately, IE9 does not seem to support it).

Core ajax upload

In order to get things going, you’ll need the following (stated also in the requirements page):

  1. jQuery 1.6 or above.
  2. jQuery UI 1.8 or above (widget factory is the minimum requirement).
  3. jQuery File Upload plugin. For the most basic implementation, you’ll only need the files jquery.fileupload.js and jquery.iframe-transport.js from that zip.

I based the following code on the “Basic Plugin” instructions. Here’s the client code:

  1. Lines 9-12 are the basic scripts required. I noticed that without the iframe-transport js file, IE doen’t upload anything. As for the jQuery and jQuery UI, you may choose to use the available CDN’s, such as Google Libraries API or Microsoft Ajax Content Delivery Network.
  2. Line 15 activates the fileupload plugin.
  3. Line 16 seems to be required in IE9 for repeated uploads. The documentation states that “By default, the file input field is replaced with a clone after each input field change event. This is required for iframe transport queues and allows change events to be fired for the same file selection, but can be disabled by setting this option to false.” On my machine IE9 failed to upload after the first attempt, unless I set this flag to false. UPDATE: On another website I’m developing, setting this flag to true or false made no difference and IE failed repeated uploads. I’m not sure why, but I just guess that there was some coincidental conflicts between this mechanism and the website’s that caused this to malfunction. What I ended up doing was replacing the file field with jQuery, each time the ‘done’ callback was called, and activated the plugin on the new file field.
  4. Line 17 states that the result is expected on json format.
  5. Line 18 sets the upload url.
  6. Lines 19-23 handle the result using the ‘done’ callback (as shown in the File Upload “basic plugin” page.
  7. Line 28 defines the html file upload with the multiple attribute (“multiple” is not required for running this sample).

The server side implementation is a standard ashx. I based it on Bo Schatzberg’s sample:

  1. Line 10 checks if there are files being uploaded. I wasn’t familiar with the context.Request.Files collection before, so thanks Bo! You may want to log and return a message if there are no files.
  2. Lines 12-14 establish a target location for the uploaded files.
  3. Line 16 gets the file. You may ask yourself why this implementation doesn’t iterate over the context.Request.Files collection to get all the files (after all, we did ask for multiple file selection and uploads). It did, originally, but then I noticed that although the files are multiply selected, they are uploaded one at a time.
  4. Lines 17-18 determine the full file name and perform the actual save.
  5. Lines 20-23 render a json response to the client. Note that line 20 contains a content type of “text/plain” instead of the anticipated “application/json”. As the FAQ states, IE displays a download dialog if the content type is “application/json” as it originally should have been. I didn’t notice this on IE9, but perhaps in an earlier IE this is the case.

Note that the ‘file’ reference in line 16 references a Stream, so you don’t necessarily have to save the file before processing it. You can just use a StreamReader or the like and process the data (thus you can remove lines 12-18 completely if you choose to do so).

There are some other Options you can tweak detailed on this page. More importantly, the options page also documents many callback events which can be used. For example, if you’d like to intercept when a file is selected, you may want to override the default implementation of the ‘add’ operation. As the documentation states, ‘add’ defaults to submitting the data using ‘data.submit( )’. But you might want to perform a validation on the selected file, for example, that the file extension is correct. So your implementation may look like this:

Although the documentation states that it’s possible to block the file selection type using “accept=’image/png’ “, I preferred to test it myself and to provide a validator-like error message. Note that this example also addresses an actual issue I came across: when I tried to use the good old validators on the file element as I always did (for example when a file selection is mandatory, or to test the file extension), they did not work as expected. It’s as if the ‘onchange event life-cycle’ was faulty when it came to the ‘add’ callback. That’s why I reverted to a Label and to checking the file extension “manually” as shown above. Another alternative may be to instruct the upload plug-in not to start uploading the files immediately, and to validate when the user manually clicks on a certain “upload” button.

Form Data

One other feature I found to be useful, is the ability to upload not only the file, but also plain data, either form element values (which is the default), or just custom data. Consider the options to upload with the file user related data, or just plain context data. I used to do this using the query string, but now this can be easily done using the ‘formData’ callback. Here’s the client code:

Here’s the server side code change:

Uploaded file size limitations

Last but not least, in ASP.NET the upload max default is 4096K, which means that you are limited in file upload size. You can change that easily in the web.config using the maxRequestLength:

If you have no control over this, you may want to check the maxChunkSize plugin option, which should be able to upload your files in chunks.

Summary

No doubt that file uploading in the web has come a long way since the days of Classic ASP and even ASP.NET’s FileUpload control. Thumbs up for Sebastian Tschan on his excellent File Upload plugin which seems to provide lots and lots of features (most of which weren’t discussed in this post at all, but are detailed here, such as resume upload and cancel upload).

 
51 Comments

Posted by on 11/09/2011 in Software Development

 

Tags: , , ,