Onenote OCR capabilities in a desktop software

Question

Is there an API to use Onenote OCR capabilities to recognise text in images automatically?

Answer 1

If you have OneNote client on the same machine as your program will execute you can create a page in OneNote and insert the image through the COM API. Then you can read the page in XML format which will include the OCR'ed text.

You want to use

1. Application.CreateNewPage to create a page
2. Application.UpdatePageContent to insert the image
3. Application.GetPageContent to read the page content and look for OCRData and OCRText elements in the XML.

OneNote COM API is documented here: http://msdn.microsoft.com/en-us/library/office/jj680120(v=office.15).aspx

Answer 2

When you put an image on a page in OneNote through the API, any images will automatically be OCR'd. The user will then be able to search any text in the images in OneNote. However, you cannot pull the image back and read the OCR'd text at this point.

If this is a feature that interests you, I invite you to go to our UserVoice site and submit this idea: http://onenote.uservoice.com/forums/245490-onenote-developers

update: vote on the idea: https://onenote.uservoice.com/forums/245490-onenote-developer-apis/suggestions/10671321-make-ocr-available-in-the-c-api

-- James

Answer 3

There is a really good sample of how to do this here: http://www.journeyofcode.com/free-easy-ocr-c-using-onenote/

The main bit of code is:

private string RecognizeIntern(Image image)
{
    this._page.Reload();

    this._page.Clear();
    this._page.AddImage(image);

    this._page.Save();

    int total = 0;
    do
    {
        Thread.Sleep(PollInterval);

        this._page.Reload();

        string result = this._page.ReadOcrText();
        if (result != null)
            return result;
    } while (total++ < PollAttempts);

    return null;
}

Answer 4

As I will be deleting my blog (which was mentioned in another post), I thought I should add the content here for future reference:

Usage

Let's start by taking a look on how to use the component: The class OnenoteOcrEngine implements the core functionality and implements the interface IOcrEngine which provides a single method:

public interface IOcrEngine
{
    string Recognize(Image image);
}

Excluding any error handling, it can be used in a way similar to the following one:

using (var ocrEngine = new OnenoteOcrEngine())
using (var image = Image.FromFile(imagePath))
{
    var text = ocrEngine.Recognize(image);
    if (text == null)
        Console.WriteLine("nothing recognized");
    else
        Console.WriteLine("Recognized: " + text);
}

Implementation

The implementation is far less straight-forward. Prior to Office 2010, Microsoft Office Document Imaging (MODI) was available for OCR. Unfortunately, this no longer is the case. Further research confirmed that OneNote's OCR functionality is not directly exposed in form of an API, but the suggestions were made to manually parse OneNote documents for the text (see Is it possible to do OCR on a Tiff image using the OneNote interop API? or need a document to extract text from image using onenote Interop?. And that's exactly what I did:

1. Connect to OneNote using COM interop
2. Create a temporary page containing the image to process
3. Show the temporary page (important because OneNote won't perform the OCR otherwise)
4. Poll for an OCRData tag containing an OCRText tag in the XML code of the page.
5. Delete the temporary page

Challenges included the parsing of the XML code for which I decided to use LINQ to XML. For example, inserting the image was done using the following code:

private XElement CreateImageTag(Image image)
{
    var img = new XElement(XName.Get("Image", OneNoteNamespace));

    var data = new XElement(XName.Get("Data", OneNoteNamespace));
    data.Value = this.ToBase64(image);
    img.Add(data);

    return img;
}

private string ToBase64(Image image)
{
    using (var memoryStream = new MemoryStream())
    {
        image.Save(memoryStream, ImageFormat.Png);

        var binary = memoryStream.ToArray();
        return Convert.ToBase64String(binary);
    }
}

Note the usage of XName.Get("Image", OneNoteNamespace) (where OneNoteNamespace is the constant "http://schemas.microsoft.com/office/onenote/2013/onenote" ) for creating the element with the correct namespace and the method ToBase64 which serializes an GDI-image from memory into the Base64 format. Unfortunately, polling (See What is wrong with polling? for a discussion of the topic) in combination with a timeout is necessary to determine whether the detection process has completed successfully:

int total = 0;
do
{
    Thread.Sleep(PollInterval);

    this._page.Reload();

    string result = this._page.ReadOcrText();
    if (result != null)
        return result;
} while (total++ < PollAttempts);

Results

The results are not perfect. Considering the quality of the images, however, they are more than satisfactory in my opinion. I could successfully use the component in my project. One issue remains which is very annoying: Sometimes, OneNote crashes during the process. Most of the times, a simple restart will fix this issue, but trying to recognise text from some images reproducibly crashes OneNote.

Code / Download

Check out the code at GitHub

Answer 5

not sure about OCR, but the documentation site for onenote API is this

http://msdn.microsoft.com/en-us/library/office/dn575425.aspx#sectionSection1