Top2OneNote Addin – Making my digital notepad notes searchable 18

Last week I bought a nice gadget at our discount supermarket (Aldi). The gadget is called a Digital notepad and makes it possible to record your notes while writing them down on paper. Your writing is stored on the internal memory as vector data. There is a lot of software supplied, but using them is not that straightforward.

Digital notepad

Most of the software supplied with this gadget wants to convert the note to clear text. But because of my bad handwriting, most of the time the converted text makes no sense. I also want to keep the drawing which I added to my notes. So instead of converting it to text, I would be happy if I can archive the notes and make them searchable.

Microsoft OneNote gives Tablet PC users the option to add handwritten notes to there OneNote notebook. Those notes are made searchable, while the handwritten note still exists. Instead of using a Tablet PC I want to convert my Digital Notepad content to OneNote handwritten notes. And therefore I developed a small OneNote addin which makes this possible. The source code together with the installer can be found at http://top2onenote.codeplex.com. Within this blogpost, I will explain some of the steps I took to make this addin possible.

Reading a TOP file

There is less information available about the file format that is generated by the Digital Notepad. The file extension is called .Top and it looks like it is a format invented by Waltop International Corp. When searching on the Internet for more information I found this blogpost together with some perl and python scripts to convert a Top file to SVG. I will use the logic of these scripts to convert my notes to OneNote.

Develop an OneNote AddIn

Visual Studio 2008 gives you the possiblity to develop Office Add Ins. There are different project templates supplied for Word, Excel and Outlook… but not for OneNote. There are no Visual Studio templates available for OneNote, so developing a AddIn is not that easy. Again Google gives me a nice search result, a blogpost about creating a toolbar addin for OneNote 2007: http://blogs.msdn.com/descapa/archive/2006/08/31/734298.aspx. There is a step by step tutorial supplied which will explain the different steps to develop a toolbar Addin which will popup a Hello world message. The AddIn will make use of COM Interop, which makes it possible to use C#.

The interface which should be implemented does only contains two methods. I found out that using this interface will only gives you the active OneNote page. With the OneNote.ApplicationClass (part of the Microsoft.Office.OneNote.Interop assembly) you will update the OneNote page outside OneNote. This means that the functionality of your addin does not run within OneNote, but inside a DllHost container.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Runtime.InteropServices;
using OneNote = Microsoft.Office.Interop.OneNote;

namespace Top2OneNoteAddIn
{
   [ComImport, Guid("C9590FA7-2132-47fb-9A78-AF0BF19AF4E6")]
   public interface IOneNoteAddIn
   {
      bool OnClick([In] String strActivePageID);

      bool OnEvent([In] OneNote.OneNoteAddIn_Event evt, [In] String strParameter);
   }
}

Note: together with OneNote 2007 I also installed the OneNote 2010 Beta. It seems like the AddIn functionality will change, because the addin is shown on the toolbar but clicking on it does not trigger the functionality. I think that with the release of OneNote 2010 that I will be easier to develop OneNote Addins, probably with the use of a Visual Studio template.

OpenFileDialog issue

The AddIn should, when clicked, open up a file dialog where the user can select the .top file which should be imported. Because the plugin does not run for within OneNote, the OpenFileDialog was shown below the OneNote window. I found a solution by supplying the Windows handler of OneNote as owner of the OpenFileDialog. This way the dialog is shown on top of OneNote

Process[] procs = Process.GetProcessesByName("OneNote");
IntPtr hwnd = procs[0].MainWindowHandle;

OpenFileDialog dialog = new OpenFileDialog();
dialog.Filter = "top files (*.top)|*.top|All files (*.*)|*.*";
dialog.Title = "Select a top file";
dialog.ShowDialog(new WindowWrapper(hwnd));

A little WindowWrapper class is added to the solution, which implements the IWin32Window interface making it possible to supply the window handler as argument to the ShowDialog method.

public class WindowWrapper : System.Windows.Forms.IWin32Window
{
        private IntPtr _hwnd;

        public WindowWrapper(IntPtr handle)
        {
            _hwnd = handle;
        }

        #region IWin32Window Members

        public IntPtr Handle
        {
            get { return _hwnd; }
        }

        #endregion
}

InkAnalyzer

The vector data from the top file is converted to a collection of Stroke objects (for which the constructor gets a StylusPointCollection as argument). Those stoke objects are set to the InkAnalyzer. The Analyze method performs layout analysis, writing and drawing classification, and handwriting recognition. The result is given back as an AnalysisStatus object which will be used for building the XML structure used to update the OneNote page.

InkAnalyzer analyzer = new InkAnalyzer();
analyzer.AddStroke(stroke);

AnalysisStatus status = analyzer.Analyze();
if (status.Successful)
{
   Console.WriteLine(analyzer.GetRecognizedString());
}

The AddStroke method has overloaded method for which you can specify the languageId of the stroke. By default the locale settings of the active thread will be used. In my case the Dutch language id was used (because of my region settings). I use English as my OS display language. When the Dutch language pack is not installed, the result of the analyzer will not be successful. So keep in mind to check if the correct language packs are installed to make handwritten recognition possible. The method analyzer.GetInkRecognizersByPriority() gives back a list of available InkRecognizers for the different languages.

Building the OneNote XML

You can update OneNote pages by providing a XML structure. Adding or updating depends if objectids are supplied. If this is the case, the object will be updated. An example of the XML used to add the handwritten notes to OneNote

<?xml version="1.0"?>
<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2007/onenote" 
        ID="{C6CCDE4D-7F47-46FA-9DD6-0A4CEC503E86}{1}{B0}">
	<one:Outline>
		<one:Position x="209.9905395507812" y="126.0"/>
		<one:Size width="245.2818908691406" height="125.3196716308594"/>
		<one:OEChildren>
			<one:OE>
				<one:InkWord recognizedText="hallo">
					<one:Data>AMIGHQSQBOYCAYABKgAaH4tCcEWEdYC+0EXZBFjPVIrml8VPjwb4utLhmyK/
7HHMzCKxQYkOuxMs7nqh3rsTi+Ves0icm6ElWrdREvqHlTLy7BdBuMAWv/Mr9HrEZm2nDbJYSZodTkA0BmwQRVOAL
HoY1USZZUo0BjFbMxSVQJ6SzJhLtWqQHBLFWlUDC0gURSNGI1cNAAAABQcLZU9mZ2hpGRQyCACAHgIh4uJBMwgA
4BICSvPiQRGrqtNBZAMVRgBAShs20YdmzeA0YdmxgA6YdmsBow7GLAB0w7Nm8AAATxMEawMVRgBAahA+gEXPAA
ARXQADggAAIqIAAAqKAW+C/gdb+B19kqxUublEssCwCSiblSgFlABLLFSyypYWLLFlliiC/gR7+BH7ACTZc3LJqblks2W
CzZQ2VNliyklZSMkzVmrKsVKSgNyygAoAESBQIxrXX3TKARMEEgARIKFVgU7sSj5OjNx+XqrpdDQFFEYAQAAABRRGA
QAAAAoAESDcOupAmua8QApiMYL+CJP4IlAAAEoAAEoAgv4BG/gEfKycSbsm5di0qry2bEGy8tUKABEg4IA52V90ygE
TBBIAESCouC8UZctFT6fx0sc+lOXuBRRGAEAAAAUURgFAAAAKP0Aj06oIUjE4DWAKWiGC/gnj+CeQBBZZQACC/gEj
+ARWM5yUt25ddnaFiwAKABEgIH932l90ygETBBIAESAf+viILJADTKojJ+vOFaLHBRRGAEAAAAUURgGAAAAJFiKA30M
Qcv/ZAAp0SYL+BFP4EVAACUALAkJRU3NypSbCwUCC/gDT+ANeJMktzPHc7Le7dtVaNy7WOIgkuJbmpZGrZQAKAB
EgYAtv1V90ygETBBIAESAUVWvkl8H9TbQgjxGuoKlABRRGAEAAAAUURgDAAAAKP0Ai63IF0ksIC+AKggFlgv4MK/gw
25SpYsFlhLAESwlsmybLNypbKAssqWUlllixZYsAgv4EY/gRkAlgZYWWFlEs2WaFUTZVkqJUuUlgAssssVAACgARIGBI
qdtfdMoBEwQSABEgk453gdolgkGItwM1/odLkQUURgBAAAAFFEYBwAAACgARIGLKJkG7kLtA
					</one:Data>
				</one:InkWord>
			</one:OE>
		</one:OEChildren>
	</one:Outline>
	<one:Outline>
		<one:Position x="225.0141754150391" y="461.2535400390625"/>
		<one:Size width="680.25830078125" height="80.7023696899414"/>
		<one:OEChildren>
			<one:OE>
				<one:InkWord recognizedText="dit">...
					</one:Data>
				</one:InkWord>
				<one:InkWord recognizedText="is">
					<one:Data>...
					</one:Data>
				</one:InkWord>
				<one:InkWord recognizedText="een">
					<one:Data>...</one:Data>
				</one:InkWord>
				<one:InkWord recognizedText="test">
					<one:Data>...
					</one:Data>
				</one:InkWord>
			</one:OE>
		</one:OEChildren>
	</one:Outline>
</one:Page>

Only the data for the first word is shown in this example to keep things clear. At the root element (Page) the ID of the existing page (provided as argument of the OnClick method) to define that the content should be inserted to the page.The InkWord data are stored as Ink Serialized Format with BASE64 encoding and placed together with the recognizedText to make searching within the handwritten notes possible.

InkWordNode iwNode = node as InkWordNode;

using (MemoryStream ms = new MemoryStream())
{
   iwNode.Strokes.Save(ms);
   byte[] isfBytes = ms.ToArray();
   dataNode.SetValue(Convert.ToBase64String(isfBytes));
}

Updating OneNote

Updating the OneNote page is done by creating an instance of the OneNote.ApplicationClass. The UpdatePageContent method is used whereby the created XML structure is supplied as argument. There is a little delay between executing the importer and when the page is updated. The cause of this delay is probally that the OneNote page is updated outside OneNote and that it will be triggered to refresh the page content.

OneNote.ApplicationClass onApplication = new OneNote.ApplicationClass();

XNamespace one = "http://schemas.microsoft.com/office/onenote/2007/onenote";
XElement root = new XElement(one + "Page", new XAttribute(XNamespace.Xmlns + "one", one));
root.Add(new XAttribute("ID", strActivePageID));

BuildXML(analyzer.RootNode, root, one);

onApplication.UpdatePageContent(root.ToString(), DateTime.MinValue);

What’s next?

The result of the importer is the same when writing the text directly with the use of a Tablet PC. Probably the results of a Tablet PC are maybe even more secure, but that’s also the reason that I don’t want to convert the handwritten notes to text.

Search within a handwritten note

Search within a handwritten note

The importer could be optimized in different ways. Some issues/ideas to improve this add in

  • Language selection – selected by import or set as a preference
  • Insert at cursor position – currently the location is set to the top left. It is possible to retrieve the cursor position by requesting the page content (GetPageContent) with the use of the OneNote.ApplicationClass. The location of the cursor is marked as a selection inside the retrieved XML structure
  • Improve text recognition – the InkAnalyzer has different properties to improve the text recognition. This way better results are achieved
  • Drag and drop support – instead of using the open file dialog, it would be nice if you can just drag the top file on your OneNote page

Like I mentioned, you can find the source at http://top2onenote.codeplex.com. It is not my goal to put a lot of effort in the development of this AddIn. If you are interested in improving this AddIn, send some feedback and I can add you as contributor to this project. If you have other questions, ask them and I will try to answer them.

18 thoughts on “Top2OneNote Addin – Making my digital notepad notes searchable

  1. Reply John Dec 8, 2009 5:25 pm

    Nice work! I just wish I knew more about the hardware involved.
    John

  2. Reply JohnforAmerica Dec 15, 2009 4:08 pm

    Nice work! Will this work with any tablet that outputs .top, or just walcom?

  3. Reply JohnforAmerica Dec 15, 2009 4:08 pm

    er, waltop…

    sorry

  4. Pingback: Latest digital notepad news – SolidTek DM-L2 DigiMemo L2 8-1/2-by-11-Inch Digital Notepad

  5. Pingback: авто новости» Архив блога » Top2OneNote AddIn

  6. Reply thomas plagwitz Apr 28, 2010 5:54 pm

    I am all about value, and this post made my day since it combines those 2 companies that traditionally have been providing the best value in my technology use… :-)

    have been holding back on purchasing a waltop, but will definitely give it a shot at the next opportunity.

    many thanks,

  7. Reply David Apr 28, 2010 6:26 pm

    Great work..
    makes the note taking more concentric now that you don’t have to use proprietry software and divide your notes between onenote and other unfurnished software.
    can you take a look at the Pulse smartpen files, it will be really great if we could convert these into onenote strokes, what can be more exciting is to also transfer the audio recording from the pen and to have it still synced in onenote. Pulse has a great product whos only downside is them insisting we use the attached software with no real export capabilities other then images.

    Thanks, David.

  8. Pingback: SharePoint Kaffeetasse 179 - Michael Greth [SharePoint MVP] - SharePointCommunity

  9. Pingback: AddIns für OneNote 2010 entwickeln « Der faule Programmierer

  10. Reply Sac Aug 7, 2010 2:07 am

    I love that you made this. However, I don’t know how to program at all and I’m not sure if I understood all of your instructions. Is there any way you could make it so that I could download the adding directly from your site so I can just install it directly, rather than remaking the whole thing on my own?

  11. Reply Lars Hanses Oct 21, 2010 8:53 am

    Thank you very much! It worked after I removed the “register for COM-Interop” option in the “build” tab. Otherwise I get always an Error message.
    But perhapse you can check your conversion from the TOP file. In some pages I missed some “vectors” especially if there are some sketches between the text.

    Many thanks,
    Lars

  12. Reply Lars Hanses Oct 22, 2010 11:01 am

    HI Johann,

    I find no email-add so I try it this way… I have a TOP file which is not imported to OneNote, but the others worked. Do you want to take a look on it? Please contact me Lars_Hanses@gmx.net

  13. Reply Jim Doria Dec 7, 2010 9:58 pm

    Thank you for creating this and making it publicly available!!

    There used to be software called CyberConverter from a company called Blue Euclid that did this, but they went off the web and their software is no longer available.

    I gave up on the idea of buying a digimemo pad because if I couldn’t use it’s ink with OneNote (as ink) I figured why bother?

    Now I can put that Adesso CyberPad back on my wish list!!

  14. Reply Lutz Jan 3, 2011 5:00 pm

    Great code, works like a charm, thanks.
    I have however problem with the Source-Code, as I always get a Type library exporter error:

    The assembly “\Top2OneNoteAddIn\bin\Debug\Top2OneNoteAddIn.dll” could not be converted to a type library. Type library exporter encountered an error while processing ‘Top2OneNoteAddIn.Top2OneNote, Top2OneNoteAddIn’. Error: Element not found.

    I already double-checked all GUIDs, but cannot find the error :S
    Any advice?

    Thanks a lot
    Lutz

  15. Reply interim Jan 11, 2011 6:27 pm

    I really enjoyed reading this post, big fan. Keep up the good work and
    please tell me when can you publish more articles or where can I read
    more on the subject?

  16. Reply Jim Doria Feb 24, 2011 8:35 pm

    Just checked back today as I just got my CyberPad today…

    Is there a binary release for this project?

    I don’t have C# installed so the source code won’t do me any good. :-(
    But I’m VERY excited to give this a try!

  17. Reply Jim Doria Mar 1, 2011 4:49 am

    It’s me again…

    I got help from a developer over at Donationcoder.com, who created the installer file you need to install this add-in.

    I posted a thread there about the whole process, including getting the add-in to work on PCs that do not have ink support built in (such as XP Professional.)

    http://www.donationcoder.com/forum/index.php?topic=25903.0

    Thanks again for sharing this great tool! I’m very happy now that I’ve got it working! :-)

Leave a Reply

  

  

  

*