English : Johan de Koning

Turing machine implementation with WF4 and FlowChart

Written on April 7, 2010 by Johan de Koning | Comment

A couple of days ago I received a tweet about a real hardware implementation of the Turing Machine (http://www.aturingmachine.com/). Impressed about this implementation, I was thinking about how to implement a simple Turing Machine counting program with the use of Workflow Foundation 4. And that’s where the story starts about my first experience with WF4.

Turing machine definition

At Wikipedia a definition of the Turing machine can be found: http://en.wikipedia.org/wiki/Turing_machine. To keep it short, a Turing machine consists of two parts

A tape of unlimited length which is separated in cells
A machine that can read/write on the tape

The machine is a finite state machine. The machine is in one state and with the use of a set of rules the machine can transfer to a different state. A rule contains the following parts

current state
symbol read
next state
symbol to write
move direction

The following rule [state: 0, read: blank, new state: 1, write: blank, move: left] will be executed by the machine when the current state is 0 and the value of the cell on the tape is blank. When the rule is executed, the machine will transfer to state 1, keeps the cell blank and will move the tape one cell to the left. A next rule for state 1 will be executed based on the new cell value read on the new tape position.

The machine part will be written with the use of a workflow diagram. This diagram will contain the different states and rule actions.

Counting rules

The rules for making counting possible (as taken from the aturingmachine.com website)

[state: 0, read: 1, new state: 0 write: 1, move: right]
[state: 0, read: 0, new state: 0, write: 0, move: right]
[state: 0, read: blank, new state: 1, write: blank, move: left]

[state: 1, read: 0, new state: 0, write: 0, move: right]
[state: 1, read: 1, new state: 1, write: 0, move: left]
[state: 1, read: blank, new state: 0, write: 1, move: right]

Tape implementation

The tape is a collection of cells which has an unlimited length. New cells can be added at the start or end of the written tape part. To support this functionality, an implementation is made based on a List collection making it possible to not only add new values at the end of the tape but also at the beginning.

public class Tape : ITape
    {
        public int Position { get; private set; }
        public int Length { get { return _internalCollection.Count; } }

        private List<int?> _internalCollection;

        public Tape()
        {
            Position = 0;
            _internalCollection = new List<int?>(){null};
        }

        public int? ReadCell()
        {
            return _internalCollection[Position];
        }

        public void WriteCell(int? cellValue)
        {
            _internalCollection[Position] = cellValue;
        }

        public void MoveCell(MoveDirection direction)
        {
            switch(direction){
                case MoveDirection.Left:
                    if (Position == 0)
                    {
                        _internalCollection.Insert(0, null);
                    }
                    else
                    {
                        Position--;
                    }
                    break;
                case MoveDirection.Right:
                    Position++;
                    if (Position == _internalCollection.Count)
                    {
                        _internalCollection.Add(null);
                    }
                    break;
            }
        }

        public override string ToString()
        {
            StringBuilder tapeValue = new StringBuilder("[");
            foreach (var cell in _internalCollection)
            {
                if (cell == null) { tapeValue.Append(" "); } else { tapeValue.Append(cell.ToString()); }
            }
            tapeValue.Append("] ");
            return tapeValue.ToString();
        }
    }

Flowchart

The state workflow of Workflow Foundation 3 is not part of version 4. The alternative given is the flowchart. With the flow chart certain types of state machine workflows can be implemented (but not all of them). A flowchart is powerful and simple to use, making it possible to loop back to previous executed activities. This is very useful for moving between the different states.

Together with the Flowchart, two activities are introducted: FlowDecision and FlowSwitch. A FlowDecision can be seen as an if condition while the FlowSwitch is like a switch statement in code. The FlowSwitch will be the central decision node for each state within the turing machine counting implementation

Workflow implementation with standard activities

A console workflow application is used for this implementation. An instance of the Tape is set as variable on the FlowChart.

The counting program can be developed with the use of the standard activities provided by WF4. The following activities are used

FlowSwitch: based on the cell value an execution path is chosen
InvokeMethod: used for executing the WriteCell and MoveCell methods
WriteLine: write the value of all cell values to the console.
Delay: this activiity is optional. Some delay to make output more readable during execution

The default path of the FlowSwitch activity is used for supporting the blank values. This is because of the fact that FlowSwitch works with string values, making it impossible to define a path for a null value.

Custom activities

The InvokeMethod activity is a generic activity to make execution of a method possible. It takes some time to configure each activity. The method parameters (for example the cell value which should be written) are implemented as a Parameter collection, which are not directly visible from the property editor within Visual Studio. To simplify/fasten this assignment process, custom activities will be developed to support the specific methods on the Tape instance.

Custom code activities are based on the CodeActivity or NativeActivity class. The CodeActivity is for creating simple synchronous custom activities. This model is suitable for the custom activities needed by the turing machine workflow. When you want to use all of the functionality exposed by the WF4 runtime, you should use the NativeActivity base class.

Two custom activities are developed

MoveCell activity: for moving the position on the tape one cell. The Direction property holds the move direction
WriteCell activity: for writing a cell value on the current position. The CellValue property holds the value which should be written

namespace TuringMachine.Activities
{
    [Designer(typeof(MoveCellDesigner))]
    public sealed class MoveCell : CodeActivity
    {
        [RequiredArgument()]
        public InArgument<ITape> Tape { get; set; }

        public MoveDirection Direction { get; set; }

        protected override void Execute(CodeActivityContext context)
        {
            var tape = context.GetValue(Tape);
            tape.MoveCell(Direction);
        }
    }
}

namespace TuringMachine.Activities
{
    [Designer(typeof(WriteCellDesigner))]
    public sealed class WriteCell : CodeActivity
    {
        [RequiredArgument()]
        public InArgument<Nullable<int>> CellValue { get; set; }

        [RequiredArgument()]
        public InArgument<ITape> Tape { get; set; }

        protected override void Execute(CodeActivityContext context)
        {
            var tape = context.GetValue(Tape);
            var cellValue = context.GetValue(CellValue);

            tape.WriteCell(cellValue);
        }
    }
}

The InArgument type used for the properties can hold (besides a value like a normal property) an expression. Those expression are defined using the VB.Net language (so null should be defined as Nothing)

The custom activities are added to the Workflow Toolbox making it possible to drag them into the workflow. You can change the property values from within the Property editor

Custom design

When viewing the diagram, it is still not directly clear which values are written and to which direction the position on the tape is moved. WF4 makes it possible to assign a design to a custom activity. The design of the activity is developed with XAML. Assignment is done with the use of the Designer attribute (as shown in the code above)

The XAML used for the MoveCell activity makes it possible to directly select the move direction from within the diagram

<sap:ActivityDesigner x:Class="TuringMachine.Activities.Designer.MoveCellDesigner"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:sap="clr-namespace:System.Activities.Presentation;assembly=System.Activities.Presentation"
    xmlns:sapv="clr-namespace:System.Activities.Presentation.View;assembly=System.Activities.Presentation"
    xmlns:sys="clr-namespace:System;assembly=mscorlib"
    xmlns:tm="clr-namespace:TuringMachine.Classes;assembly=TuringMachine.Classes">
    <sap:ActivityDesigner.Resources>
        <ObjectDataProvider MethodName="GetValues" ObjectType="{x:Type sys:Enum}" x:Key="MoveDirectionValues">
            <ObjectDataProvider.MethodParameters>
                <x:TypeExtension TypeName="tm:MoveDirection"/>
            </ObjectDataProvider.MethodParameters>
        </ObjectDataProvider>
    </sap:ActivityDesigner.Resources>
    <Grid>
        <Grid.ColumnDefinitions>
            <ColumnDefinition Width="auto"/>
            <ColumnDefinition Width="*"/>
        </Grid.ColumnDefinitions>
        <Grid.RowDefinitions>
            <RowDefinition Height="*"/>
        </Grid.RowDefinitions>
        <TextBlock VerticalAlignment="Center">Direction:</TextBlock>
        <ComboBox Grid.Column="1" ItemsSource="{Binding Source={StaticResource MoveDirectionValues}}" SelectedValue="{Binding ModelItem.Direction, Mode=TwoWay}"></ComboBox>
    </Grid>
</sap:ActivityDesigner>

And the WriteCell Activity makes the cell value assignment possible from within the workflow diagram. For the WriteCell activity an ExpressionTextBox is used, supporting the InArgument expression possibilities.

<sap:ActivityDesigner x:Class="TuringMachine.Activities.Designer.WriteCellDesigner"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:sap="clr-namespace:System.Activities.Presentation;assembly=System.Activities.Presentation"
    xmlns:sapv="clr-namespace:System.Activities.Presentation.View;assembly=System.Activities.Presentation"
    xmlns:conv="clr-namespace:System.Activities.Presentation.Converters;assembly=System.Activities.Presentation"
    xmlns:local="clr-namespace:TuringMachine.Activities.Designer"
    xmlns:sys="clr-namespace:System;assembly=mscorlib">
  <sap:ActivityDesigner.Resources>
    <conv:ArgumentToExpressionConverter x:Key="expressionConverter"/>
  </sap:ActivityDesigner.Resources>
  <Grid>
        <Grid.ColumnDefinitions>
            <ColumnDefinition Width="auto"/>
            <ColumnDefinition Width="*"/>
        </Grid.ColumnDefinitions>
        <Grid.RowDefinitions>
            <RowDefinition Height="*"/>
        </Grid.RowDefinitions>
        <TextBlock VerticalAlignment="Center">Cell value</TextBlock>
        <sapv:ExpressionTextBox Grid.Column="1" Expression="{Binding Path=ModelItem.CellValue, Mode=TwoWay, ConverterParameter=In, Converter={StaticResource expressionConverter}}" ExpressionType="{local:NullableExtension sys:Int32}" OwnerActivity="{Binding Path=ModelItem}"></sapv:ExpressionTextBox>
    </Grid>
</sap:ActivityDesigner>

Workflow implementation with custom activities

With the use of the developed custom activities together with their design, the diagram of the workflow will give us directly more information related to the execution of the program

What’s next?

My first impression about WF4 is that the usage is more simplified. Within the previous version of WF you had to write a lot of code to execute external methods. With the use of the CodeActivity (or NativeActivity) class it is easy to implement custom activities. The possibility to use XAML to layout your custom activities is very powerful. It makes it possible to implement diagrams which are visual more meaningful (especially useful when hosting the WF designer within your own application). While the FlowChart is not a real State Workflow, it is suitable for the implementation written above. The usage feels more natural, just like developing a sequence workflow.

While developing the counting workflow I had some issues with the support of Nullable types and the ExpressionTextBox. I needed to implement an TypeExtension to make it possible to define the ExpressionType as a Nullable. This was not what I expected because XAML 2009 (XAML version of .Net 4) should make it possible to define the type as {x:Type TypeName=System:Nullable`1[[System.Int32]]}. But this syntax is not accepted as valid.

Source code

The visual studio 2010 solution including the workflows and source code files can be found at (http://johandekoning.codeplex.com -> browse to WF/TuringMachine)

Read more from the Uncategorized category. If you would like to leave a comment, click here: Comment. or stay up to date with this post via RSS, or you can Trackback from your site.

Powershell – Sync/shift file creation time

Written on March 29, 2010 by Johan de Koning | Comment

I went on holiday for two weeks together with my girlfriend. We both had our own camera and when we returned home we wanted to merge both photo collections. There was only a small problem, the date/time settings of the cameras were not in sync. This will give you funny/strange results when viewing the slideshow.

To solve this issue, I created a small powershell script. As an argument you provide a datetime string (format ddMMyyyyHHmmss). The oldest file (inside the directory where you execute this script) will get this new provided creation date. The creation date of the other files is updated taken the same time span (between oldest file date and the provided date) into account. Executing this script on one of the photo collections will make it possible to merge both collections.

$oldestFile = dir | sort-object CreationTime | select -first 1

$newStartDateTime = [datetime]::ParseExact($args,"ddMMyyyyHHmmss",$null)
$timespan = $newStartDateTime - $oldestFile.CreationTime

dir | foreach { $_.CreationTime = ($_.CreationTime + $timespan)}

Read more from the Uncategorized category. If you would like to leave a comment, click here: Comment. or stay up to date with this post via RSS, or you can Trackback from your site.

My colleague went to a conference and all I got was this lousy eco-button

Written on January 14, 2010 by Johan de Koning | 2 Comments

Today I got an eco button from my colleague. It is a hardware button with a green flashing light (is that eco?) which can be connected to your computer. With software installed, your computer will be put in stand-by mode when you press the button.

Because I use my computer the whole day for development, I don’t need an eco-button. I need an build project button for the Visual Studio development environment. A little hack is really easy to make and the result will be a more impressive build routine

Pressing the button (after connecting it to your computer without installing the ecobutton software) will execute the keys command

Windows-Key+R ecobutton ENTER

In my opinion a stupid implementation, because each time you press the button you see a Run dialog, followed by the “ecobutton” text filled in and finished with an enter. But this way it is quite easy to replace ecobutton.exe with your own application (keeping the same name)

So I developed a little console application (maybe a hidden application is even better) which will search my Visual Studio application and send the key command F6 (you can also send CTRL+SHIFT+B).

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Runtime.InteropServices;
using System.Diagnostics;

namespace WhatDoesThisButtonDo
{
    class Program
    {
        [DllImport("user32.dll")]
        static extern bool SetForegroundWindow(IntPtr hWnd);

        static void Main(string[] args)
        {
            IntPtr hWnd = IntPtr.Zero;

            foreach (var p in Process.GetProcesses())
            {
                if (p.ProcessName.Equals("devenv")) {
                    hWnd = p.MainWindowHandle;
                    break;
                }
            }

            if (hWnd != IntPtr.Zero)
            {
                SetForegroundWindow(hWnd);
                SendKeys.SendWait("{F6}");
            }
        }
    }
}

Explaining the code

When pressing the ecobutton, Visual Studio is not the active application anymore. Therefore I first have to find the window handler (of Visual Studio), make it the active foreground application and send the F6 key. There is a FindWindow method (user32.dll) but this one is based on the caption on the title bar. Because your solution name is part of this caption, the name is not static. To solve this issue I loop through the different processes to find the devenv process (when multiple visual studio’s are running, only one is used in this sample).

The SetForegroundWindow brings Visual Studio to the foreground. The F6 key is sent with SendKeys.SendWait. Instead of the F6 key you can also send the CTRL+SHIFT+B keystrokes by supplying “^+B”

To use this small applicaton, make sure it is called ecobutton.exe and placed within your PATH (otherwise Windows is unable to find it).

For me the Build button is way better than the Eco button version, but probaly you have better/other implementation suggestions what you can do with this hardware device. Please post them as feedback on this blogpost.

Read more from the Uncategorized category. If you would like to leave a comment, click here: 2 Comments. or stay up to date with this post via RSS, or you can Trackback from your site.

Top2OneNote Addin – Making my digital notepad notes searchable

Written on December 8, 2009 by Johan de Koning | 11 Comments

Last week I bought a nice gadget at our discount supermarket (Aldi). The gadget is called a Digital notepad and makes it possible to record your notes while writing them down on paper. Your writing is stored on the internal memory as vector data. There is a lot of software supplied, but using them is not that straightforward.

Most of the software supplied with this gadget wants to convert the note to clear text. But because of my bad handwriting, most of the time the converted text makes no sense. I also want to keep the drawing which I added to my notes. So instead of converting it to text, I would be happy if I can archive the notes and make them searchable.

Microsoft OneNote gives Tablet PC users the option to add handwritten notes to there OneNote notebook. Those notes are made searchable, while the handwritten note still exists. Instead of using a Tablet PC I want to convert my Digital Notepad content to OneNote handwritten notes. And therefore I developed a small OneNote addin which makes this possible. The source code together with the installer can be found at http://top2onenote.codeplex.com. Within this blogpost, I will explain some of the steps I took to make this addin possible.

Reading a TOP file

There is less information available about the file format that is generated by the Digital Notepad. The file extension is called .Top and it looks like it is a format invented by Waltop International Corp. When searching on the Internet for more information I found this blogpost together with some perl and python scripts to convert a Top file to SVG. I will use the logic of these scripts to convert my notes to OneNote.

Develop an OneNote AddIn

Visual Studio 2008 gives you the possiblity to develop Office Add Ins. There are different project templates supplied for Word, Excel and Outlook… but not for OneNote. There are no Visual Studio templates available for OneNote, so developing a AddIn is not that easy. Again Google gives me a nice search result, a blogpost about creating a toolbar addin for OneNote 2007: http://blogs.msdn.com/descapa/archive/2006/08/31/734298.aspx. There is a step by step tutorial supplied which will explain the different steps to develop a toolbar Addin which will popup a Hello world message. The AddIn will make use of COM Interop, which makes it possible to use C#.

The interface which should be implemented does only contains two methods. I found out that using this interface will only gives you the active OneNote page. With the OneNote.ApplicationClass (part of the Microsoft.Office.OneNote.Interop assembly) you will update the OneNote page outside OneNote. This means that the functionality of your addin does not run within OneNote, but inside a DllHost container.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Runtime.InteropServices;
using OneNote = Microsoft.Office.Interop.OneNote;

namespace Top2OneNoteAddIn
{
   [ComImport, Guid("C9590FA7-2132-47fb-9A78-AF0BF19AF4E6")]
   public interface IOneNoteAddIn
   {
      bool OnClick([In] String strActivePageID);

      bool OnEvent([In] OneNote.OneNoteAddIn_Event evt, [In] String strParameter);
   }
}

Note: together with OneNote 2007 I also installed the OneNote 2010 Beta. It seems like the AddIn functionality will change, because the addin is shown on the toolbar but clicking on it does not trigger the functionality. I think that with the release of OneNote 2010 that I will be easier to develop OneNote Addins, probably with the use of a Visual Studio template.

OpenFileDialog issue

The AddIn should, when clicked, open up a file dialog where the user can select the .top file which should be imported. Because the plugin does not run for within OneNote, the OpenFileDialog was shown below the OneNote window. I found a solution by supplying the Windows handler of OneNote as owner of the OpenFileDialog. This way the dialog is shown on top of OneNote

Process[] procs = Process.GetProcessesByName("OneNote");
IntPtr hwnd = procs[0].MainWindowHandle;

OpenFileDialog dialog = new OpenFileDialog();
dialog.Filter = "top files (*.top)|*.top|All files (*.*)|*.*";
dialog.Title = "Select a top file";
dialog.ShowDialog(new WindowWrapper(hwnd));

A little WindowWrapper class is added to the solution, which implements the IWin32Window interface making it possible to supply the window handler as argument to the ShowDialog method.

public class WindowWrapper : System.Windows.Forms.IWin32Window
{
        private IntPtr _hwnd;

        public WindowWrapper(IntPtr handle)
        {
            _hwnd = handle;
        }

        #region IWin32Window Members

        public IntPtr Handle
        {
            get { return _hwnd; }
        }

        #endregion
}

InkAnalyzer

The vector data from the top file is converted to a collection of Stroke objects (for which the constructor gets a StylusPointCollection as argument). Those stoke objects are set to the InkAnalyzer. The Analyze method performs layout analysis, writing and drawing classification, and handwriting recognition. The result is given back as an AnalysisStatus object which will be used for building the XML structure used to update the OneNote page.

InkAnalyzer analyzer = new InkAnalyzer();
analyzer.AddStroke(stroke);

AnalysisStatus status = analyzer.Analyze();
if (status.Successful)
{
   Console.WriteLine(analyzer.GetRecognizedString());
}

The AddStroke method has overloaded method for which you can specify the languageId of the stroke. By default the locale settings of the active thread will be used. In my case the Dutch language id was used (because of my region settings). I use English as my OS display language. When the Dutch language pack is not installed, the result of the analyzer will not be successful. So keep in mind to check if the correct language packs are installed to make handwritten recognition possible. The method analyzer.GetInkRecognizersByPriority() gives back a list of available InkRecognizers for the different languages.

Building the OneNote XML

You can update OneNote pages by providing a XML structure. Adding or updating depends if objectids are supplied. If this is the case, the object will be updated. An example of the XML used to add the handwritten notes to OneNote

<?xml version="1.0"?>
<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2007/onenote"
        ID="{C6CCDE4D-7F47-46FA-9DD6-0A4CEC503E86}{1}{B0}">
	<one:Outline>
		<one:Position x="209.9905395507812" y="126.0"/>
		<one:Size width="245.2818908691406" height="125.3196716308594"/>
		<one:OEChildren>
			<one:OE>
				<one:InkWord recognizedText="hallo">
					<one:Data>AMIGHQSQBOYCAYABKgAaH4tCcEWEdYC+0EXZBFjPVIrml8VPjwb4utLhmyK/
7HHMzCKxQYkOuxMs7nqh3rsTi+Ves0icm6ElWrdREvqHlTLy7BdBuMAWv/Mr9HrEZm2nDbJYSZodTkA0BmwQRVOAL
HoY1USZZUo0BjFbMxSVQJ6SzJhLtWqQHBLFWlUDC0gURSNGI1cNAAAABQcLZU9mZ2hpGRQyCACAHgIh4uJBMwgA
4BICSvPiQRGrqtNBZAMVRgBAShs20YdmzeA0YdmxgA6YdmsBow7GLAB0w7Nm8AAATxMEawMVRgBAahA+gEXPAA
ARXQADggAAIqIAAAqKAW+C/gdb+B19kqxUublEssCwCSiblSgFlABLLFSyypYWLLFlliiC/gR7+BH7ACTZc3LJqblks2W
CzZQ2VNliyklZSMkzVmrKsVKSgNyygAoAESBQIxrXX3TKARMEEgARIKFVgU7sSj5OjNx+XqrpdDQFFEYAQAAABRRGA
QAAAAoAESDcOupAmua8QApiMYL+CJP4IlAAAEoAAEoAgv4BG/gEfKycSbsm5di0qry2bEGy8tUKABEg4IA52V90ygE
TBBIAESCouC8UZctFT6fx0sc+lOXuBRRGAEAAAAUURgFAAAAKP0Aj06oIUjE4DWAKWiGC/gnj+CeQBBZZQACC/gEj
+ARWM5yUt25ddnaFiwAKABEgIH932l90ygETBBIAESAf+viILJADTKojJ+vOFaLHBRRGAEAAAAUURgGAAAAJFiKA30M
Qcv/ZAAp0SYL+BFP4EVAACUALAkJRU3NypSbCwUCC/gDT+ANeJMktzPHc7Le7dtVaNy7WOIgkuJbmpZGrZQAKAB
EgYAtv1V90ygETBBIAESAUVWvkl8H9TbQgjxGuoKlABRRGAEAAAAUURgDAAAAKP0Ai63IF0ksIC+AKggFlgv4MK/gw
25SpYsFlhLAESwlsmybLNypbKAssqWUlllixZYsAgv4EY/gRkAlgZYWWFlEs2WaFUTZVkqJUuUlgAssssVAACgARIGBI
qdtfdMoBEwQSABEgk453gdolgkGItwM1/odLkQUURgBAAAAFFEYBwAAACgARIGLKJkG7kLtA
					</one:Data>
				</one:InkWord>
			</one:OE>
		</one:OEChildren>
	</one:Outline>
	<one:Outline>
		<one:Position x="225.0141754150391" y="461.2535400390625"/>
		<one:Size width="680.25830078125" height="80.7023696899414"/>
		<one:OEChildren>
			<one:OE>
				<one:InkWord recognizedText="dit">...
					</one:Data>
				</one:InkWord>
				<one:InkWord recognizedText="is">
					<one:Data>...
					</one:Data>
				</one:InkWord>
				<one:InkWord recognizedText="een">
					<one:Data>...</one:Data>
				</one:InkWord>
				<one:InkWord recognizedText="test">
					<one:Data>...
					</one:Data>
				</one:InkWord>
			</one:OE>
		</one:OEChildren>
	</one:Outline>
</one:Page>

Only the data for the first word is shown in this example to keep things clear. At the root element (Page) the ID of the existing page (provided as argument of the OnClick method) to define that the content should be inserted to the page.The InkWord data are stored as Ink Serialized Format with BASE64 encoding and placed together with the recognizedText to make searching within the handwritten notes possible.

InkWordNode iwNode = node as InkWordNode;

using (MemoryStream ms = new MemoryStream())
{
   iwNode.Strokes.Save(ms);
   byte[] isfBytes = ms.ToArray();
   dataNode.SetValue(Convert.ToBase64String(isfBytes));
}

Updating OneNote

Updating the OneNote page is done by creating an instance of the OneNote.ApplicationClass. The UpdatePageContent method is used whereby the created XML structure is supplied as argument. There is a little delay between executing the importer and when the page is updated. The cause of this delay is probally that the OneNote page is updated outside OneNote and that it will be triggered to refresh the page content.

OneNote.ApplicationClass onApplication = new OneNote.ApplicationClass();

XNamespace one = "http://schemas.microsoft.com/office/onenote/2007/onenote";
XElement root = new XElement(one + "Page", new XAttribute(XNamespace.Xmlns + "one", one));
root.Add(new XAttribute("ID", strActivePageID));

BuildXML(analyzer.RootNode, root, one);

onApplication.UpdatePageContent(root.ToString(), DateTime.MinValue);

What’s next?

The result of the importer is the same when writing the text directly with the use of a Tablet PC. Probably the results of a Tablet PC are maybe even more secure, but that’s also the reason that I don’t want to convert the handwritten notes to text.

Search within a handwritten note

The importer could be optimized in different ways. Some issues/ideas to improve this add in

Language selection – selected by import or set as a preference
Insert at cursor position – currently the location is set to the top left. It is possible to retrieve the cursor position by requesting the page content (GetPageContent) with the use of the OneNote.ApplicationClass. The location of the cursor is marked as a selection inside the retrieved XML structure
Improve text recognition – the InkAnalyzer has different properties to improve the text recognition. This way better results are achieved
Drag and drop support – instead of using the open file dialog, it would be nice if you can just drag the top file on your OneNote page

Like I mentioned, you can find the source at http://top2onenote.codeplex.com. It is not my goal to put a lot of effort in the development of this AddIn. If you are interested in improving this AddIn, send some feedback and I can add you as contributor to this project. If you have other questions, ask them and I will try to answer them.

Read more from the Uncategorized category. If you would like to leave a comment, click here: 11 Comments. or stay up to date with this post via RSS, or you can Trackback from your site.

XMPP and WCF .Net – Connecting a XMPP client to Openfire

Written on December 1, 2009 by Johan de Koning | Comment

With Openfire installed (http://www.johandekoning.nl/index.php/2009/11/30/xmpp-and-wcf-net-installation-of-openfire/) it is time to create some users and connect with a XMPP client to the server.

User management

Because of the simple installation, users created on the Openfire server are stored inside the embedded database. To add new users, make sure that the server runs and open the administration console (default address http://localhost:9090).

Note: I already mentioned this in my previous blog. Within my Vista environment I had to make sure that I start the Openfire server as administrator

Within the Administration Console, you will find the Users/Groups section at the top of the page. When you open this section you will get an overview of the users created on the Openfire server

User Summary

Click on the Create New User option on the left, to create a new user. The Create User page needs some information for the new user. Provide this information and click create user to add the user. Or choose Create & Create another if you want to create another user after this user account is added.

Create User

I create two users which are called user1 and user2. I gave the the same password as their username. The overview shows the following user list.

User Summary (users created)

One of the core extensions of the XMPP protocol is the ability to have a user list which is stored on the server (which is called a Roster). Within the Administration Console you can manage those lists, to make it possible that user1 has user2 as a contact and visa versa. This is off course also possible from within a XMPP client where you can add new users.

To manage a contacts list (roster) for a user, click on the username (of the user you want to manage) from the overview. You will get the User Properties screen, with some basic information.

User properties

At the left you will find a Roster option. When you click that option, you get an overview of the roster items. These items can be other users, or groups of users. Click the Add New Item (at the right of the screen) to add user 2 to the roster items of user 1.

Add roster item

When added, the user roster overview will show user 2. Perform the same action for user 2 but this time add user 1 to the roster list.

User Roster

When both users are on both list, the user configuration part is finished. I know it is very basic, but this is jus for development. When can now use a XMPP client to make a connection with our server.

Using Psi (XMPP Client)

There are a lot of XMPP clients available on the Internet. I use Psi because it has a nice XML logging feature, to make the XMPP stream visible. I can use this sample stream for development of a XMPP client in .Net without the use of WCF. You can download the Psi client at http://psi-im.org/download/

When you start the client for the first time, it will show you a message that you need to set up an account. Click on the Use existing account option, because our test accounts are already available on the server.

Account setup

On the first tab (Account) fill in the Jabber ID (JID) and password. The jabber ids for our users are user1@localfire and user2@localfire . The password are user1 and user2.

Account Tab

At the connection tab, we need to specify our server location. This is localhost when it runs on the same machine where you installed this client. For testing purposes we will not use an encrypt connection (Never).

Connection tab

Click save to store the account information. The main screen of Psi will be visible whereby the user status is set to Offline.

Psi main screen

Before we connect to our Openfire server, I want to enable the XML console to get an idea of the stream which is send between client and server. Right click on the first item of the list (openfire) and select Open XML Console… Within the XML Console, mark the enable checbox at the bottem left. To get the client connected, select from the dropdownlist at the bottom of the main screen, the Online status. The client will connect to the server and the stream will be made visible in the XML Console dialog.

XML Console

The XMPP stream

I connected another client to the server as user 2. Between both users i send a message with the text “Hello” (User 1 -> User 2) and “Goodbye” (User 2 -> User 1). The presence of user 1 will be set to Offline and the application will be closed. Below you find the captured XMPP stream.

What’s next?

This blog post explained how to set up the connecting between a XMPP client and server. The captured stream will be used for further analysis. The next step will be the development of a simple XMPP client which will echo back received messages in reverse order. This client will not be written with Windows Communication Foundation to get some feeling how a .Net client with the TcpClient class will work.

If you have questions after reading this post, or have other feedback? Please send in a comment.

<!--client-->
<?xml version="1.0"?>
<stream:stream xmlns:stream="http://etherx.jabber.org/streams" version="1.0" xmlns="jabber:client" to="localfire" xml:lang="en" xmlns:xml="http://www.w3.org/XML/1998/namespace" >

<!--server-->
<?xml version='1.0' encoding='UTF-8'?><stream:stream xmlns:stream="http://etherx.jabber.org/streams" xmlns="jabber:client" from="localfire" id="c035b19c" xml:lang="en" version="1.0">

<stream:features>
<starttls xmlns="urn:ietf:params:xml:ns:xmpp-tls"/>
<mechanisms xmlns="urn:ietf:params:xml:ns:xmpp-sasl">
<mechanism>DIGEST-MD5</mechanism>
<mechanism>PLAIN</mechanism>
<mechanism>ANONYMOUS</mechanism>
<mechanism>CRAM-MD5</mechanism>
</mechanisms>
<compression xmlns="http://jabber.org/features/compress">
<method>zlib</method>
</compression>
<auth xmlns="http://jabber.org/features/iq-auth"/>
<register xmlns="http://jabber.org/features/iq-register"/>
</stream:features>

<!--client-->
<auth xmlns="urn:ietf:params:xml:ns:xmpp-sasl" mechanism="DIGEST-MD5" />

<!--server-->
<challenge xmlns="urn:ietf:params:xml:ns:xmpp-sasl">cmVhbG09ImxvY2FsZmlyZSIsbm9uY2U9IitaQVRkeUZNeWJjV1ZveHJHTXNIV2dxNVR1QWhNUkFoQ08wVWlp
VzciLHFvcD0iYXV0aCIsY2hhcnNldD11dGYtOCxhbGdvcml0aG09bWQ1LXNlc3M=</challenge>

<!--client-->
<response xmlns="urn:ietf:params:xml:ns:xmpp-sasl">dXNlcm5hbWU9InVzZXIxIixyZWFsbT0ibG9jYWxmaXJlIixub25jZT0iK1pBVGR5Rk15YmNXVm94ckdNc0hXZ3E1
VHVBaE1SQWhDTzBVaWlXNyIsY25vbmNlPSJhM0dMY21ra3VFUjEwaVBTMlJzQnpRZWlBY2pzSnIzNVNyTzRkTFNzM
2FnPSIsbmM9MDAwMDAwMDEsZGlnZXN0LXVyaT0ieG1wcC9sb2NhbGZpcmUiLHFvcD1hdXRoLHJlc3BvbnNlPWM1N
TlhMjBhMmFkZGVjYmJjOTFkZTZkNDYwODYyODU5LGNoYXJzZXQ9dXRmLTg=</response>

<!--server-->
<success xmlns="urn:ietf:params:xml:ns:xmpp-sasl">cnNwYXV0aD04NjMxMTIzNDZlM2MxOTUxMGZlNzFmNTA5N2E0MWViMg==</success>

<!--client-->
<?xml version="1.0"?>
<stream:stream xmlns:stream="http://etherx.jabber.org/streams" version="1.0" xmlns="jabber:client" to="localfire" xml:lang="en" xmlns:xml="http://www.w3.org/XML/1998/namespace" >

<!--server-->
<?xml version='1.0' encoding='UTF-8'?><stream:stream xmlns:stream="http://etherx.jabber.org/streams" xmlns="jabber:client" from="localfire" id="c035b19c" xml:lang="en" version="1.0">
<stream:features>
<compression xmlns="http://jabber.org/features/compress">
<method>zlib</method>
</compression>
<bind xmlns="urn:ietf:params:xml:ns:xmpp-bind"/>
<session xmlns="urn:ietf:params:xml:ns:xmpp-session"/>
</stream:features>

<!--client-->
<iq type="set" id="bind_1" >
<bind xmlns="urn:ietf:params:xml:ns:xmpp-bind">
<resource>Home</resource>
</bind>
</iq>

<!--server-->
<iq xmlns="jabber:client" type="result" id="bind_1" to="localfire/c035b19c" >
<bind xmlns="urn:ietf:params:xml:ns:xmpp-bind">
<jid>user1@localfire/Home</jid>
</bind>
</iq>

<!--client-->
<iq type="set" id="aab7a" >
<session xmlns="urn:ietf:params:xml:ns:xmpp-session"/>
</iq>

<!--server-->
<iq type="result" id="aab7a" to="user1@localfire/Home" >
<session xmlns="urn:ietf:params:xml:ns:xmpp-session"/>
</iq>

<!--client-->
<iq type="get" id="aab8a" >
<query xmlns="jabber:iq:roster"/>
</iq>

<!--server-->
<iq type="result" id="aab8a" to="user1@localfire/Home" >
<query xmlns="jabber:iq:roster">
<item subscription="both" jid="user2@localfire" />
</query>
</iq>

<!--client-->
<presence>
<priority>5</priority>
<c xmlns="http://jabber.org/protocol/caps" node="http://psi-im.org/caps" ver="0.13-dev-rev2" ext="ca cs ep-notify html" />
</presence>

<iq type="get" to="user1@localfire/Home" id="aabaa" >
<query xmlns="http://jabber.org/protocol/disco#info" node="http://psi-im.org/caps#ca" />
</iq>

<iq type="get" id="aabba" >
<query xmlns="jabber:iq:private">
<storage xmlns="storage:bookmarks"/>
</query>
</iq>

<iq type="get" to="user1@localfire" id="aabca" >
<vCard xmlns="vcard-temp" version="2.0" prodid="-//HandGen//NONSGML vGen v1.0//EN" />
</iq>

<iq type="get" to="localfire" id="aabda" >
<query xmlns="http://jabber.org/protocol/disco#info"/>
</iq>

<!--server-->
<presence from="user2@localfire/Work" to="user1@localfire/Home" >
<c xmlns="http://jabber.org/protocol/caps" node="http://exodus.jabberstudio.org/caps" ext="xhtml-im" ver="0.10.0.0" />
<priority>1</priority>
</presence>

<!--client-->
<iq type="get" to="user2@localfire/Work" id="aabea" >
<query xmlns="http://jabber.org/protocol/disco#info" node="http://exodus.jabberstudio.org/caps#0.10.0.0" />
</iq>

<iq type="get" to="user2@localfire/Work" id="aabfa" >
<query xmlns="http://jabber.org/protocol/disco#info" node="http://exodus.jabberstudio.org/caps#xhtml-im" />
</iq>

<!--server-->
<iq from="user1@localfire/Home" type="get" to="user1@localfire/Home" id="aabaa" >
<query xmlns="http://jabber.org/protocol/disco#info" node="http://psi-im.org/caps#ca" />
</iq>

<!--client-->
<iq type="result" to="user1@localfire/Home" id="aabaa" >
<query xmlns="http://jabber.org/protocol/disco#info" node="http://psi-im.org/caps#ca" >
<identity category="client" type="pc" name="Psi" />
<feature var="urn:xmpp:jingle:1" />
<feature var="urn:xmpp:jingle:transports:ice-udp:1" />
<feature var="urn:xmpp:jingle:apps:rtp:1" />
<feature var="urn:xmpp:jingle:apps:rtp:audio" />
</query>
</iq>

<!--server-->
<iq type="result" id="aabba" to="user1@localfire/Home" >
<query xmlns="jabber:iq:private">

<storage xmlns="storage:bookmarks"/>
</query>
</iq>

<iq from="user1@localfire" type="result" id="aabca" to="user1@localfire/Home" >
<vCard xmlns="vcard-temp" version="2.0" prodid="-//HandGen//NONSGML vGen v1.0//EN" >
<FN>User 1</FN>
</vCard>
</iq>

<iq from="localfire" type="result" id="aabda" to="user1@localfire/Home" >
<query xmlns="http://jabber.org/protocol/disco#info">
<identity category="server" type="im" name="Openfire Server" />
<identity category="pubsub" type="pep" />
<feature var="http://jabber.org/protocol/pubsub#manage-subscriptions" />
<feature var="http://jabber.org/protocol/pubsub#modify-affiliations" />
<feature var="http://jabber.org/protocol/pubsub#retrieve-default" />
<feature var="http://jabber.org/protocol/pubsub#collections" />
<feature var="jabber:iq:private" />
<feature var="http://jabber.org/protocol/disco#items" />
<feature var="vcard-temp" />
<feature var="http://jabber.org/protocol/pubsub#publish" />
<feature var="http://jabber.org/protocol/pubsub#subscribe" />
<feature var="http://jabber.org/protocol/pubsub#retract-items" />
<feature var="http://jabber.org/protocol/offline" />
<feature var="http://jabber.org/protocol/pubsub#meta-data" />
<feature var="jabber:iq:register" />
<feature var="http://jabber.org/protocol/pubsub#retrieve-subscriptions" />
<feature var="http://jabber.org/protocol/pubsub#default_access_model_open" />
<feature var="jabber:iq:roster" />
<feature var="http://jabber.org/protocol/pubsub#config-node" />
<feature var="http://jabber.org/protocol/address" />
<feature var="http://jabber.org/protocol/pubsub#publisher-affiliation" />
<feature var="http://jabber.org/protocol/pubsub#item-ids" />
<feature var="http://jabber.org/protocol/pubsub#instant-nodes" />
<feature var="http://jabber.org/protocol/commands" />
<feature var="http://jabber.org/protocol/pubsub#multi-subscribe" />
<feature var="http://jabber.org/protocol/pubsub#outcast-affiliation" />
<feature var="http://jabber.org/protocol/pubsub#get-pending" />
<feature var="google:jingleinfo" />
<feature var="jabber:iq:privacy" />
<feature var="http://jabber.org/protocol/pubsub#subscription-options" />
<feature var="jabber:iq:last" />
<feature var="http://jabber.org/protocol/pubsub#create-and-configure" />
<feature var="urn:xmpp:ping" />
<feature var="http://jabber.org/protocol/pubsub#retrieve-items" />
<feature var="jabber:iq:time" />
<feature var="http://jabber.org/protocol/pubsub#create-nodes" />
<feature var="http://jabber.org/protocol/pubsub#persistent-items" />
<feature var="jabber:iq:version" />
<feature var="http://jabber.org/protocol/pubsub#presence-notifications" />
<feature var="http://jabber.org/protocol/pubsub" />
<feature var="http://jabber.org/protocol/pubsub#retrieve-affiliations" />
<feature var="http://jabber.org/protocol/pubsub#delete-nodes" />
<feature var="http://jabber.org/protocol/pubsub#purge-nodes" />
<feature var="http://jabber.org/protocol/disco#info" />
<feature var="http://jabber.org/protocol/rsm" />
</query>
</iq>

<iq from="user2@localfire/Work" type="result" id="aabea" to="user1@localfire/Home" >
<query xmlns="http://jabber.org/protocol/disco#info" node="http://exodus.jabberstudio.org/caps#0.10.0.0" >
<feature var="jabber:iq:agents" />
<feature var="jabber:iq:oob" />
<feature var="jabber:iq:browse" />
<feature var="jabber:iq:time" />
<feature var="jabber:iq:version" />
<feature var="jabber:iq:last" />
<feature var="http://jabber.org/protocol/disco#items" />
<feature var="http://jabber.org/protocol/disco#info" />
<feature var="storage:bookmarks" />
<feature var="jabber:x:data" />
<feature var="jabber:x:conference" />
<feature var="jabber:x:event" />
<feature var="http://jabber.org/protocol/muc" />
<feature var="http://jabber.org/protocol/muc#user" />
<feature var="http://jabber.org/protocol/muc#owner" />
<feature var="http://jabber.org/protocol/si" />
<feature var="http://jabber.org/protocol/si/profile/file-transfer" />
<feature var="http://jabber.org/protocol/bytestreams" />
</query>
</iq>

<iq from="user2@localfire/Work" type="result" id="aabfa" to="user1@localfire/Home" >
<query xmlns="http://jabber.org/protocol/disco#info" node="http://exodus.jabberstudio.org/caps#xhtml-im" >
<feature var="http://jabber.org/protocol/xhtml-im" />
</query>
</iq>

<iq from="user1@localfire/Home" type="result" to="user1@localfire/Home" id="aabaa" >
<query xmlns="http://jabber.org/protocol/disco#info" node="http://psi-im.org/caps#ca" >
<identity category="client" type="pc" name="Psi" />
<feature var="urn:xmpp:jingle:1" />
<feature var="urn:xmpp:jingle:transports:ice-udp:1" />
<feature var="urn:xmpp:jingle:apps:rtp:1" />
<feature var="urn:xmpp:jingle:apps:rtp:audio" />
</query>
</iq>

<!--client-->
<message type="chat" to="user2@localfire" id="aac0a" >
<body>Hello</body>
<active xmlns="http://jabber.org/protocol/chatstates"/>
</message>

<!--server-->
<message from="user2@localfire/Work" type="chat" id="jcl_13" to="user1@localfire/Home" >
<body>Goodbye</body>
<x xmlns="jabber:x:event">
<composing/>
</x>
<html xmlns="http://jabber.org/protocol/xhtml-im">
<body xmlns="http://www.w3.org/1999/xhtml">
<p style="font-size:small;font-family:Arial;color:#000000" >Goodbye</p>
</body>
</html>
</message>

<!--client-->
<presence type="unavailable" >
<status>Logged out</status>
</presence>

</stream:stream>

<!--server-->
</stream:stream>

Read more from the Uncategorized category. If you would like to leave a comment, click here: Comment. or stay up to date with this post via RSS, or you can Trackback from your site.

XMPP and WCF .Net – Installation of OpenFire

Written on November 30, 2009 by Johan de Koning | 1 Comment

In my previous blogpost, I talked about my goal to make an XMPP implementation within .Net based on Windows Communication Foundation. Before I can start, I had to find out how a normal XMPP stream would be send/retrieved between client and server.

In this blogitem, I will explain the steps I took to install the OpenFire server. http://www.igniterealtime.org/projects/openfire/index.jsp Within my other blog I used a seperate environment which is mostly based on Ubuntu. This time I install the OpenFire server within Windows Vista. The reason for this is that it is also the environment of my .Net development tools.

Installation wizard

First download the installation file of OpenFire from the website. I use Openfire 3.6.4 http://www.igniterealtime.org/downloads/index.jsp

The OpenFire website

Download page

The installation wizard is quite straight forward and more like next-next-finish.

Language selection

Welcome step of the wizard

License Agreement

Installation location

Start menu shortcut

Progress of installation

Completing wizard step

On the last step, mark the Run Openfire option (which is marked by default) to launch the Openfire server. This makes it possible to continue the installation from the Admin Console (which runs inside the browser).

Openfire server status

Admin console installation/configuration

To finalize the installation, some setup steps will be shown within the Admin Console. Select the prefered language in the first step

Language selection

You can keep the server settings values the default. Because I will only use the Openfire server locally (or within my own netwerk) it is not important that the right domain is set. The clients will connect to localhost or the ipaddress of the computer.

Server settings

Because I want to keep the installation as simple as possible, I choose for an Embedded database. The performance penalty with the embedded database is for test purposes not an issue.

Database settings

I will create the test users from within the Admin console. This means that I don’t need Active Directory/LDAP or Clearspace integration. I choose for the Default option to store users inside the server database.

Profile settings

To access the admin console after these setup steps, it is important to setup an Administration account. The username of this account will be “admin”

Administrator account

This completes the openfire setup process. You can now login to the real admin console

Note: I had some trouble to log in to the Admin Console on my Vista enviroment. The admin user was not valid. I found out that I had to restart the Openfire server by stopping the server, quit the status dialog and start the Openfire server again for the start menu. It is important to start the Openfire server as administrator (right click on the shortcut and select Run as administrator) otherwise you are still not able to login.

Admin console login

Server information page

What’s next?

As a mentioned the installation of Openfire, with the (default) settings I chose, is easy and straight forward. The next step will be to configure some users and to connect a XMPP client to the server. In the meantime I really like to have you feedback on this blogpost (did you like the images to explain the installation) or on my other blogposts.

Read more from the Uncategorized category. If you would like to leave a comment, click here: 1 Comment. or stay up to date with this post via RSS, or you can Trackback from your site.

XMPP and WCF .Net – Introduction

Written on November 29, 2009 by Johan de Koning | 1 Comment

Hummm… XMPP and WCF? Is this not a blog page about Android or other Google related stuff. That’s partly true but as a Microsoft .Net developer I am more interested in combining the best of both world. I like Google and Microsoft products. I really like the open source vision of Google. But when it comes to development tools I prefer using Microsoft.

So that’s me, living between both worlds. Using the best of both worlds to develop good implementations. And that is where this new blog post are about.

Some time ago Google launched Google Wave presentation (http://wave.google.com/help/wave/about.html) When I watched the presentation, I was (and still am) impressed.

At my previous job, we developed a real time location aware application. A huge amount of data was collected by the location systems, was processed by the server and was made visible on the clients. The made visible part is a tricky one, because we wanted to make use of protocols which where cross platform like SOAP. The result was a polling mechanism which sends requests on interval basis to the server to get new data.

We knew that this was not the best implementation. At that time (almost 3 years ago) we were also thicking about as a possible solution. XMPP stands for Extensible Messaging and Presence Protocol (http://xmpp.org/). A protocol which was developed already in 1999 and is based on XML. I think that the popularity of this protocol was getting a boost when Google launched their Google Talk client in augustus 2005.

I found a presentation on the Web about XMPP (http://www.slideshare.net/kellan/beyond-rest) which has a nice slide (shown in the image below). This reminds me of an episode of the Smurfs (http://en.wikipedia.org/wiki/The_Smurfs) where the Smurfs are on a long trip.

Brainy Smurf: Is it much further, Papa Smurf?
Papa Smurf: Not much further my little smurfs.
Jokey Smurf: (different location) Is it much further, Papa Smurf?
Papa Smurf: Not much further my little smurfs.
Grouchy Smurf: (third location) Is it much further, Papa Smurf?
Papa Smurf: (exasperated) Yes…it is!

Are we there yet?

So when all Smurfs are requesting data from Papa Smurf, Papa Smurf becomes overloaded. A better solution would be a Publish Subscribe implementation (http://en.wikipedia.org/wiki/Publish/subscribe). XMPP has such extension: http://xmpp.org/extensions/xep-0060.html. This way the clients (little smurfs) don’t become bored (they can do other things) and Papa Smurfs don’t get distracted or even overloaded.

Tell me when I get there

The protocol of Google Wave is based on this model. The protocol makes use of the XMPP protocol together with some extensions. Installation document of the Google Wave Federation Prototype server can be found at http://code.google.com/p/wave-protocol/wiki/Installation Of course Google is using Java or Phyton for the example clients, and that is exactly the part where my love for Microsoft fits in. I am really interested in building a Google Wave protocol client within .Net

Because XMPP is involved, the first step would be to develop a XMPP client in .Net. There are some XMPP libraries available for .Net but those don’t have a flexible extension model. And when looking at Windows Communication Foundation, there are already some parts available for setting up the connection and send/retrieve messages. Developing an XMPP library based on the WCF technology is the first goal of these new blog posts.

What’s next?

Before looking at a WCF implemtation, it is good to know how XMPP works. Therefore I will develop a sample client which makes use of the .Net TcpClient to initiate and send/retrieve messages (no WCF involved). To test this client I will install a XMPP server called OpenFire (http://www.igniterealtime.org/projects/openfire/index.jsp). Check this blog often for new updates… and remember Papa Smurf, we are not there yet!

BTW I love to retrieve comments on this (or my other blogposts). Your feedback is really important!

Read more from the Uncategorized category. If you would like to leave a comment, click here: 1 Comment. or stay up to date with this post via RSS, or you can Trackback from your site.

Building Chromium OS – update

Written on November 25, 2009 by Johan de Koning | 1 Comment

Today (or yesterday late) I started download and building the source of Chromium OS again. As a described inside my blog and at http://groups.google.com/group/chromium-os-discuss/browse_thread/thread/ba8ddf13edece9a6/647d2632e8c2d615?lnk=raot I got a blue gradient screen after login.

One of the reponse of this post said something about the different version of the gtk library which are used http://code.google.com/p/chromium-os/issues/detail?id=676 Ubuntu 9.10 makes use of libgtk2.0 2.18.3 while Chrome OS is packaged with 2.17.2. So building the code went fine but when running you get errors inside the log file var/log/slim.log

/opt/google/chrome/chrome symbol lookup error: /opt/google/chrome/ chrome undefined symbol: gdk_x11_window_get_drawable_impl

You don’t get this issue when using the pre built browser binary (because this one is compiled with the right package). But my idea was that I wanted to build the Chromium OS completly from source. Because I did not want to make another chroot environment for building the browser code, I created a new virtual machine (within VMWare) and this time I use Ubuntu 9.04

About Chromium

I updated my previous blog post and this time I get a browser after login. Still the system is far away from stable (but this is probally because it is a very early version) and performs not that smooth (probally lack of drivers). But it is working.

So check out the previous (updated) blog post (http://www.johandekoning.nl/index.php/2009/11/23/building-chromium-os/) and I hope it helps you when building the system by your own. Of course it is possible to ask question or give feedback by posting comments.

Read more from the Uncategorized category. If you would like to leave a comment, click here: 1 Comment. or stay up to date with this post via RSS, or you can Trackback from your site.

Building Chromium OS

Written on November 23, 2009 by Johan de Koning | 7 Comments

Note: for my first build I used Ubuntu 9.10. It looks like there are some issues with using this version so for my second build I am using Ubuntu 9.04. Some of the part of this blogpos became obsolete because those issues don’t exists anymore. I marked those part as red. When updating this blog I am using the revision 33052 of the browser code. I don’t know which revision I am using of the OS code

While I have no time to complete the build proces for Android 1.6 (looking at the Google application part), I quickly start looking today at building the Chromium OS (for which the source code was released today (20 november 2009)). Within this blog post, I will summarize the steps I took to build the Chromium OS. I used the steps described at http://www.chromium.org/chromium-os/building-chromium-os

Installation of Ubuntu

Just like Android, I prefer to use an Ubuntu environment as development/build environment. Android could not be build on a Windows OS, it looks like this is possible for Chromium OS. But still setting up an Ubuntu environment is quite easy. I use VMWare server to make an Ubuntu virtual machine.

With Android I used Ubuntu 8.04. This time I am using the Ubuntu 9.04 version. The document suggests Ubuntu 9.10, but it looks like that building the browser part is not completly possible because of the gtk library used (details at http://code.google.com/p/chromium-os/issues/detail?id=676). Installation of Ubuntu is executing a next-next-finish Wizard, keeping things as default as possible (partition table) and supplying some information when needed (user login).

After installation of Ubuntu, I always perform a sudo apt-get update and sudo apt-get upgrade to get the latest version of the installed packages.

Because I am using VMWare, I also install the VMWare tools. Because the VMWare tools supplied by VMWare server are not compatible with Ubuntu 9.10, I choose the Open VMWare version by following the instructions which are listed here http://langui.sh/2009/10/05/ubuntu-9-10-in-vmware/

I tried to installed the Open VMWare tools (like it did on my first attempt building Chromium with 9.10) but it looks like Open VMWare is not compatible with 9.04. I tried the normal VMWare tools supplied by VMWare server and those are working fine.

Prerequisites

To build Chromium, several libraries are needed to compile the source code. There is a package list shown at http://code.google.com/p/chromium/wiki/LinuxBuildInstructionsPrerequisites

To simplify the installation of those packages, there is a script supplied which will automatically install the requires software. This script can be found at http://src.chromium.org/svn/trunk/src/build/install-build-deps.sh Save this script on you local harddrive at a location which is easy to find/remember (for example your home directory). To execute this script, use the following commands from a terminal window

chmod a+x install-build-deps.sh
sudo ./install-build-deps.sh

This script installs all tools and libraries needed to build Chromium.
For most of the libraries, it can also install debugging symbols, which
will allow you to debug code in the system libraries. Most developers
won't need these symbols.
Do you want me to install them for you (y/N)
Skipping installation of debugging symbols.

The script will ask if you want the debugging symbols to be installed. I choose No because at this moment I don’t want to debug the code of the system libraries.

The code repostitory is not only on SVN but makes also use of git. So you also have to install the git-core package

sudo apt-get install git-core

Issue with msstcorefonts

With ubuntu 9.04 I did not have any issues with installin msstcorefonts

The install_build_deps.sh script will also install the msttcorefonts package. Installation of this package did not succeed because of (dns) time outs (the files are downloaded from sourceforge). To get rid of those timeouts, change the file at /var/lib/dpkg/info/ttf-mscorefonts-installer.postinst. Remove the -dns-timeout=10 part at line 148.

if ! wget --continue --tries=1 --connect-timeout=15 --read-timeout=300 $QUIET_ARG --directory-prefix . --no-directories --no-background $URLROOT$ff ; then

Getting the browser source code

The Chromium OS has two components: the browser and the operating system. Because, although that is my idea, you want to build the complete solution, we also download the source of the browser part to get everything. This step is optional because you can also use a pre-built binary version of this Chromium-based browser (more details at http://sites.google.com/a/chromium.org/dev/chromium-os/building-chromium-os/getting-the-chromium-os-source-code)

Install depot_tools scripts

To get the browser source, a package of scripts (depot_tools) need to be installed. Those script are used to manage checkouts and perform code review. One of the tools used inside this package is gclient to retrieve the Chromium code. The depot_tools can be retrieved in multiple ways (SVN or as a download) I use the SVN method by executing the method (from the home directory ~/)

svn co http://src.chromium.org/svn/trunk/tools/depot_tools

This will create a directory (together with the scripts) called depot_tools inside your home folder (~/depot_tools). Add this location to your PATH. I add this to the .profile file

gedit ~/.profile

Add to the the end of the file

export PATH=$HOME/depot_tools:$PATH

Save and exit the text editor. To activate this change (without rebooting), reload the .profile file (from the ~/ location)

. .profile

Check your PATH by executing the echo $PATH command. You should get something like this

/home/jcdekoning/depot_tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

Get the source code

Create a directory inside your home location called chromium. This location will hold the source which belongs to the browser (part of the OS).

mkdir ~/chromium
cd ~/chromium

Code retrieval is done with the gclient tool. Configure this tool by executing

gclient config http://src.chromium.org/svn/trunk/src http://chromium-status.appspot.com/lkgr

The first url is the SVN location. The second url is a reference to the revision number which should be downloaded (SVN works with revision numbers). The http://chromium-status.appspot.com/lkgr holds a reference to the revision which succesfully passed the unit tests (in debug mode). This way you will download not the latest version, but a version that builds (most of the time).

You can also make use of another revision reference http://build.chromium.org/buildbot/continuous/LATEST/REVISION. This is a reference to the revision that passes both unit tests and layout tests. This version will probally an older version than the lkgr one, because layout tests will take some time to run on the server.

It is also possible to specify a different revision number (by supplying a revisionnumber) but this is not part of this blog post and you will probally want to use the LKGR one. The status of the source tree can be found at http://build.chromium.org/buildbot/waterfall/console (when it is green/open you should be able to compile the code)

Retrieve the source by executing the following command. Code retrieval will take some time

gclient sync --deps="unix,chromeos"

Because you will use the code for building the Chromium OS, you need to supply the deps argument

Getting the OS source

Create a directory for the OS source. I use a location within my home directory called chromiumos

mkdir ~/chromiumos
cd ~/chromiumos

Code retrieval from the Git repository is also done with the gclient (and will also take some time)

gclient config http://src.chromium.org/git/chromiumos.git
gclient sync

Build Chromium OS

After the code is retrieved for both parts (browser and OS), it is time to build the Chromium OS. The different build scripts are inside the ~/chromiumos/chromiumos.git/src/scripts directory

cd ~/chromiumos/chromiumos.git/src/scripts

Local package repository

Chromium OS builds only it a seperate environment (chroot environment). This way the build result does not (or less) depend on the, in our case, Linux environment installed. To prepare this chroot environment, the first step will be the retrieval of the different packages which will be used to create this environment.

From the scripts directory, execute the following command to make a local repository of the packages. This will take some time

./make_local_repo.sh

Create chroot environment

After the local repository is created, it is time to set up the chroot enviroment

./make_chroot.sh

Build browser

The Chromium OS will be build within the seperate chroot environment. But before we can start to build the OS, you need to build the browser. This step is not needed when you use the binary pre-built version of the browser.

When building the browser, I got two build issues. I solved them by making some changes in the code. So before building the browser from the source, you have to make some changes.

When building the browser code (revision 33052) I did not get this issues written below. It is possible that the source code is updated (patches are merged inside revision) or that the issues are related to package version (like the gcc). So fixing both issues can be skipped.

Patch 414025

When building the source, I got an error like ‘<anonymous>’ is used uninitialized in this function inside the cookies_tree_model.cc file. This issue can be fixed by applying a patch (details at http://codereview.chromium.org/414025/show)

Download the patch file issue414025_1_2.diff from http://codereview.chromium.org/download/issue414025_1_2.diff and save it inside the ~/chromium/src/ directory. Apply the patch by running

patch -p0 -i issue414025_1_2.diff

Conflict_resolver.cc issue

Another build issue is related to the conflict_resolver.css file (~/chromium/chrome/browser/sync/engine/conflict_resolver.cc). This error is related to the fact that the compiler has some difficulties with understanding the type std::string. This issue can be solved by adding an #include <string> to the file.

Open ~/chromium/chrome/browser/sync/engine/conflict_resolver.cc inside your favorite text editor and add the following string (somewhere at the beginning of the file, i.e under #include <map>)

#include <string>

Save the file and exit the texteditor

Build the browser code

After both issues fixed, it is time to build the browser code. This is done by executing

./build_chrome.sh --chrome_dir ~/chromium

Whereby the –chrome_dir argument references to the source location of the browser code. Compiling/building will take some time.

Enter chroot build environment

Because the Chromium OS is build within its own build environment, it is time to enter this prepared enviroment. Run

./enter_chroot.sh

Chromium OS configuration

Inside the chroot environment there are two configuration steps which can be taken to make it easier to work with the Chromium OS (within this development stage).

Enable a local user account

When the Chromium OS boots, you get a login screen where you enter your Google account username and password. It is possible (and I confirm that this was the case for me) that authentication is not working (no network connection, lack of network driver support). By creating a local test user, you are able to login with this username (and empty password). To create this local user account (from within the chroot environment):

( cd ../platform/pam_google && ./enable_localaccount.sh USERNAME )

Change USERNAME to the username you want to use.

Shared user password

When you want to enter a terminal from within the Chromium OS and you want to perm task as sudo, you have to type in a password. This password is normally random generated (for security). You can define your own sudo password by executing (from within the chroot environment):

./set_shared_user_password.sh

Build the packages and kernel

Next step will be building the packages and kernel for Chromium OS. Both is done from within the chroot enviroment by executing the following commands:

./build_platform_packages.sh
./build_kernel.sh

Instead of running above commands, you can also execute ./build_all.sh. This will not only build the platform packages and kernel, but will also create the Chromium OS image (which can be copied to an USB key or VMWare/VirtualBox). When runnin build_all.sh, you can skip the next step

Building the image

From within the chroot environment, create the Chromium OS image by executing

./build_image.sh

The script will end with something like

Done.  Image created in /home/jcdekoning/trunk/src/build/images/999.999.32609.064708-a1
To copy to USB keyfob, outside the chroot, do something like:
./image_to_usb.sh --from=~/chromeos/src/build/images/999.999.32609.064708-a1 --to=/dev/sdb
To convert to VMWare image, outside the chroot, do something like:
./image_to_vmware.sh --from=~/chromeos/src/build/images/999.999.32609.064708-a1

The output directory where the image will be stored, is generated by the script. This number will probally be different on your environment.

The Chromium OS is now successfully build and you should be able to copy and run it from an USB key or use it within VMWare. Using the image is explained in the following steps

Using the image

Exit chroot

Before using the image (copy to USB or use within VMWare), you can check the image content by mounting it locally. Make sure that you exit the chroot enviroment first by running the exit command. You will be back in your own system environment.

Check the package list

Mount the image by running the following commands. Replace SUBDIR with the unique generated number for your build (i.e. 999.999.32609.064708-a1)

cd ~/chromiumos/src/build/images/SUBDIR
sudo mount -o loop rootfs.image rootfs
sudo chroot rootfs

Get a list of the installed packages by running

dpkg -l

Exit the image and unmount the image (when finished exploring the image content)

exit
sudo umount rootfs

Copy image to a USB key

Because at the moment I don’t have a 4GB USB key available, I did not performed those steps yet (details at http://sites.google.com/a/chromium.org/dev/chromium-os/building-chromium-os/build-instructions). Copying the image to the USB key is done by running the command

I got an USB key (8GB) and copyied the image by running the command

./image_to_usb.sh

This command will copy the latest build to the device /dev/sdb. If you want to copy not the latest build or if the location of the USB key is different, run the following command

./image_to_usb.sh --from=~/chromiumos/src/build/images/SUBDIR --to=/dev/USBKEYDEV

Whereby the SUBDIR should be replaced by the unique generated number, and the USBKEYDEV to the device for the USB key.

I will post my results (update this blogpost) when I performed the copy to USB key steps.

Use/convert image for VMWare

The Chromium OS image can also be used within VMWare by converting it to an vmware disk (.vmdk). You can also use this disk within Sun VirtualBox.

To make converting possible, you should install the qemu package

sudo apt-get install qemu

After the qemu package is installed, run the image_to_vmware.sh script.

./image_to_vmware.sh

This will convert the latest build to the ~/chromiumos/chromiumos.git/src/build/images/SUBDIR/ide.vmdk location (SUBDIR is the unique generated number for the build). If you want to specify the build and or the location where the VMWare image should be stored, then use the command:

./image_to_vmware.sh --from=~/chromiumos/chromiumos.git/src/build/images/SUBDIR \ --to=~/chromiumos/chromiumos.git/src/build/images/SUBDIR/ide.vmdk

I copied this image to my Vista enviroment (to get it outside Ubuntu) and created a new Virtual Machine within VMWare Server. As operating type, I selected Ubuntu. Don’t create a new disk but use an existing one (the ide.vmdk file). Make sure that you change the disk options/type to IDE (and not SCSI).

When running ChromiumOS within VMWare, I got the login screen after a couple of seconds. I am not able to login with my Google account (get a message that there is no network connecting). Login with the local user account (with empty password) is possible but after login it shows nothing (only a blue gradient screen). I hope I can solve this issue soon (first need to know what causes this issue). I will also try to arrange some USB key, to check if it is VMWare related

With my new build (25 november) it is still not possible to login with my google account. Login with the local account is possible and this time a browser is visible. Still the system is slow (except from the boot part), especially the internet connection (takes more than 10 seconds to load a page) and also the systems completly freeze at certain times (so you should perform a reboot).

Chromium OS login screen

And some new screens (after logged in).

Main menu

Twitter inside Chromium OS

What’s next…

I succesfully build the ChromiumOS (browser and OS) but am not able to use the system after login. I will try the USB key installation first to see if the blue gradient screen issue also occurs there. To solve my issue, I started a thread at the Chromium discussion group. Because this is my first post in this group, the moderator should first check my post (so it is not there yet) http://groups.google.com/group/chromium-discuss

Hope to give an update / solve this issue soon. Please let me know if you encounter the same problems

Update 23 november 2009: my post is accepted and can be found at http://groups.google.com/group/chromium-os-discuss/browse_thread/thread/ba8ddf13edece9a6# I got a response that somebody else had the same issue. So still not solved…

Update 25 november 2009: I am able to succesfully build the Chromium OS and it is also possible to use it. The system is not responding very smoothly and freezes often. Also the internet connection is really slow. One of the problems could be the lack of driver support. I also booted the OS from the USB key, but it still performs slow.

At the moment I am really impressed about the boot time. In a couple of seconds VMWare/PC shows the boot screen. Love to see this fast boot in other distributions so that it can be used for media center functionality.

It is not my plan for now to get involved in the development of this OS. So the next step for me will be to check new revisions at interval basis to get an idea about the progress made. I am also interested in the future of this OS to see whats the big thing behind this OS and if it will be used by a large group of people.

Read more from the Uncategorized category. If you would like to leave a comment, click here: 7 Comments. or stay up to date with this post via RSS, or you can Trackback from your site.

Running Android on other hardware

Written on August 16, 2009 by Johan de Koning | 5 Comments

Now that building Donut for Dream is not working at the moment (because of the prioritary files) and it is unkown if it will be possible (because of the memory size on the device) I become interested in porting Android to different hardware.

It would be nice to have hardware with the complete specifications. This way there is no (or less) problem with proprietary files because code can be written to implement these parts (because of the hardware specification available). This would be great to expand my knowledge about Android (and the linux kernel) which is currently not that big.

I was looking at some hardware and found two options: the beagleboard or the chumby. Both devices don’t have a GSM radio stack but maybe this can be added by the use of an USB device.

Beagleboard

The USB-powered Beagle Board is a low-cost, fan-less single board computer that unleashes laptop-like performance and expandability without the bulk, expense, or noise of typical desktop machines.

Above is the description of the beagleboard (http://www.beagleboard.org) as written on their website. It is a really small piece of hardware (about 7.6 cm) and can be extended with other (USB) hardware. Just like the Dream it also uses an ARM processor.

Beagle Board

A small shopping list of hardware that should be bought when using the BeagleBoard can be found at http://code.google.com/p/beagleboard/wiki/BeagleBoardShoppingList Looking at the minimal hardware list, the total price is about $250 without shipping costs(with the current exchangerate of 1.4 this will be 176 EUR). You can lower the price if you buy some parts on eBay (for example the USB HUB with Ethernet http://www.supermediastore.com/gwc-usb-2-0-3-port-hub-ethernet-adapter.html can be found on eBay for 12.50 EUR). BTW with the hardware you don’t have a nice touchscreen.

Beagleboard with connected hardware

Chumby

The chumby (http://www.chumby.com) is some kind of cute device which can be described as an Internet radio/media player/alarm clock. Because it is a complete product there is already software installed. The Chumby runs linux and makes use of widgets (which are developed in Flash).

Looking at the hardware, the Chumby has an ARM processor, 3.5 inch touchscreen, Wifi connection and sensors (Squeeze and accelerometer). The hardware specification and source code of the Chumby is available (http://www.chumby.com/developers) which makes it a really good starting point for porting Android to the Chumby.

Chumby

The price of a Chumby is $199.95 (141 EUR) shipping costs not included. The only problem for me is that I cannot order it to my address in The Netherlands.

What’s next?

I know that there is already some porting done (like for the Beagleboard http://labs.embinux.org/index.php/Android_Porting_Guide_to_Beagle_Board). But let’s be clear, I am more interested in where to start with porting Android and which steps to take than simply run some build scripts. If I know the details of the porting process than I can describe them here as a reference for other devices. BTW I am not trying to reinvent the wheel, so if some porting is done and for me it is clear why that has been done, i might/will use that ported stuff. But it should be clear why and not because somebody else said so.

When looking at the BeagleBoard and the Chumby, my choice goes to the Chumby. In the mean time, while ordering the hardware, I would like to get some pointers of you where to start with porting Android.

Read more from the Uncategorized category. If you would like to leave a comment, click here: 5 Comments. or stay up to date with this post via RSS, or you can Trackback from your site.

During the day he is a Microsoft .Net Developer, at night he is looking at Google

Turing machine definition

Counting rules

Tape implementation

Flowchart

Workflow implementation with standard activities

Custom activities

Custom design

Workflow implementation with custom activities

What’s next?

Source code

Explaining the code

Reading a TOP file

Develop an OneNote AddIn

OpenFileDialog issue

InkAnalyzer

Building the OneNote XML

Updating OneNote

What’s next?

User management

Using Psi (XMPP Client)

The XMPP stream

What’s next?

Installation wizard

Admin console installation/configuration

What’s next?

What’s next?

Installation of Ubuntu

Prerequisites

Issue with msstcorefonts

Getting the browser source code

Install depot_tools scripts

Get the source code

Getting the OS source

Build Chromium OS

Local package repository

Create chroot environment

Build browser

Patch 414025

Conflict_resolver.cc issue

Build the browser code

Enter chroot build environment

Chromium OS configuration

Enable a local user account

Shared user password

Build the packages and kernel

Building the image

Using the image

Exit chroot

Check the package list

Copy image to a USB key

Use/convert image for VMWare

What’s next…

Beagleboard

Chumby

What’s next?

About me

Twitter Updates

Tags

Recent Posts

Recent Comments

Links