17

I'm trying to use protobuf in a C# project, using protobuf-net, and am wondering what is the best way to organise this into a Visual Studio project structure.

When manually using the protogen tool to generate code into C#, life seems easy but it doesn't feel right.

I'd like the .proto file to be considered to be the primary source-code file, generating C# files as a by-product, but before the C# compiler gets involved.

The options seem to be:

  1. Custom tool for proto tools (although I can't see where to start there)
  2. Pre-build step (calling protogen or a batch-file which does that)

I have struggled with 2) above as it keeps giving me "The system cannot find the file specified" unless I use absolute paths (and I don't like forcing projects to be explicitly located).

Is there a convention (yet) for this?


Edit: Based upon @jon's comments, I retried the pre-build step method and used this (protogen's location hardcoded for now), using Google's address-book example:

c:\bin\protobuf\protogen "-i:$(ProjectDir)AddressBook.proto" 
       "-o:$(ProjectDir)AddressBook.cs" -t:c:\bin\protobuf\csharp.xslt

Edit2: Taking @jon's recommendation to minimise build-time by not processing the .proto files if they haven't changed, I've knocked together a basic tool to check for me (this could probably be expanded to a full Custom-Build tool):

using System;
using System.Diagnostics;
using System.IO;

namespace PreBuildChecker
{
    public class Checker
    {
        static int Main(string[] args)
        {
            try
            {
                Check(args);
                return 0;
            }
            catch (Exception e)
            {
                Console.WriteLine(e.Message);
                return 1;
            }
        }

        public static void Check(string[] args)
        {
            if (args.Length < 3)
            {
                throw new ArgumentException(
                    "Command line must be supplied with source, target and command-line [plus options]");
            }

            string source = args[0];
            string target = args[1];
            string executable = args[2];
            string arguments = args.Length > 3 ? GetCommandLine(args) : null;

            FileInfo targetFileInfo = new FileInfo(target);
            FileInfo sourceFileInfo = new FileInfo(source);
            if (!sourceFileInfo.Exists) 
            {
                throw new ArgumentException(string.Format(
                    "Source file {0} not found", source));
            }

            if (!targetFileInfo.Exists || 
                sourceFileInfo.LastWriteTimeUtc > targetFileInfo.LastAccessTimeUtc)
            {
                Process process = new Process();
                process.StartInfo.FileName = executable;
                process.StartInfo.Arguments = arguments;
                process.StartInfo.ErrorDialog = true;

                Console.WriteLine(string.Format(
                     "Source newer than target, launching tool: {0} {1}",
                     executable,
                     arguments));
                process.Start();
            }
        }

        private static string GetCommandLine(string[] args)
        {
            string[] arguments = new string[args.Length - 3];
            Array.Copy(args, 3, arguments, 0, arguments.Length);
            return String.Join(" ", arguments);
        }
    }
}

My pre-build command is now (all on one line):

$(SolutionDir)PreBuildChecker\$(OutDir)PreBuildChecker 
    $(ProjectDir)AddressBook.proto 
    $(ProjectDir)AddressBook.cs 
    c:\bin\protobuf\protogen 
      "-i:$(ProjectDir)AddressBook.proto" 
      "-o:$(ProjectDir)AddressBook.cs" 
      -t:c:\bin\protobuf\csharp.xslt
Marc Gravell
  • 1,026,079
  • 266
  • 2,566
  • 2,900
Ray Hayes
  • 14,896
  • 8
  • 53
  • 78
  • Side note: The guys doing the "MassTransit" project have a GPB implementation for .NET that uses a Fluent API for describing and configuring your layout. It looks much easier to use than this does. – chadmyers Jan 17 '09 at 22:06
  • I can't believe I missed this one! d'oh!!! If you still can't get this working, drop me a line. I wrote protobuf-net, so I can probably help. – Marc Gravell Jan 18 '09 at 17:07
  • Added: http://code.google.com/p/protobuf-net/issues/detail?id=39 – Marc Gravell Jan 18 '09 at 17:15
  • Just wanna comment on Edit2. This thing can be easily done without any "checker" tool just using MSBuild transforms and dependency analysis. See this link: http://msdn.microsoft.com/en-us/library/ms171476.aspx – Michal Levý Nov 19 '12 at 08:34

6 Answers6

8

As an extension of Shaun's code, I am pleased to announce that protobuf-net now has Visual Studio integration by way of a Custom Tool. The msi installer is available from the project page. More complete information here: protobuf-net; now with added Orcas.

Visual Studio with protobuf-net as a Custom Tool

Community
  • 1
  • 1
Marc Gravell
  • 1,026,079
  • 266
  • 2,566
  • 2,900
8

Calling a pre-build step but using project variables (e.g. $(ProjectPath)) to create absolute filenames without having them actually in your solution would seem a reasonable bet to me.

One thing you might want to consider, based on my past experience of code generators: you might want to write a wrapper for protogen which generates code to a different location, then checks whether the newly generated code is the same as the old code, and doesn't overwrite it if so. That way Visual Studio will realise nothing's changed and not force that project to be rebuilt - this has cut build times dramatically for me in the past.

Alternatively, you could keep an md5 hash of the .proto file the last time protogen was executed, and only execute protogen if the .proto file has changed - even less to do on each build!

Thanks for raising this as a question though - it clearly suggests I should work out a way to make this an easy pre-build step for my own port.

Jon Skeet
  • 1,421,763
  • 867
  • 9,128
  • 9,194
  • Thanks, I must have mistyped something before on the pre-build step. It would be nice if VisualStudio could do all of the "make" style checks regarding cs being older than proto files! – Ray Hayes Jan 17 '09 at 19:43
  • Yes, that would indeed be nice :) There may be a way of getting it to do that, but it would probably require more MSBuild-fu than I have, I'm afraid. The force is stronger with Marc in that respect, so he may well have a better answer. – Jon Skeet Jan 17 '09 at 19:49
  • @jon, I've now added a pre-build and a simple tool to check if the "written" date/time for the .proto is newer than the .cs, if so, it calls the protogen command. – Ray Hayes Jan 17 '09 at 20:47
  • oh, by added, I mean updated the original post with an "edit" block... not sure what the correct way to do this is.... – Ray Hayes Jan 17 '09 at 20:47
6

Add the following pre-build event to your project settings to generate the C# file only when the .proto file has changed. Just replace YourFile with the name of base name of your .proto file.

cd $(ProjectDir) && powershell -Command if (!(Test-Path YourFile.proto.cs) -or (Get-Item YourFile.proto).LastWriteTimeUtc -gt (Get-Item YourFile.proto.cs).LastWriteTimeUtc) { PathToProtoGen\protogen -i:YourFile.proto -o:YourFile.proto.cs }

This works in any recent version of Visual Studio, unlike the protobuf-net Custom-Build tool, which does not support Visual Studio 2012 or Visual Studio 2013, according to issues 338 and 413.

Edward Brey
  • 40,302
  • 20
  • 199
  • 253
6

Add this to the relevant project file.

Advantage, incremental build.

Disadvantage, you need to edit manually when adding files.

<ItemGroup>
    <Proto Include="Person.proto" />
    <Compile Include="Person.cs">
        <DependentUpon>Person.proto</DependentUpon>
    </Compile>
</ItemGroup>
<PropertyGroup>
    <CompileDependsOn>ProtobufGenerate;$(CompileDependsOn)</CompileDependsOn>
</PropertyGroup>
<Target Name="ProtobufGenerate" Inputs="@(Proto)" Outputs="@(Proto->'$(ProjectDir)%(Filename).cs')">
    <ItemGroup>
        <_protoc Include="..\packages\Google.Protobuf.*\tools\protoc.exe" />
    </ItemGroup>
    <Error Condition="!Exists(@(_protoc))" Text="Could not find protoc.exe" />
    <Exec Command="&quot;@(_protoc)&quot; &quot;--csharp_out=$(ProjectDir.TrimEnd('\'))&quot; @(Proto->'%(Identity)',' ')" WorkingDirectory="$(ProjectDir)" />
</Target>
Tamir Daniely
  • 1,659
  • 1
  • 20
  • 24
  • This excellent answer! One note it seems to expect that protoc is located somewhere in "..\packages\Google.Protobuf" which was not in my case. But if protoc is avalialble on the path, the whole _protoc ItemGroup can be ommited, and protoc can be directly referenced in Exec node. – Davorin Ruševljan May 13 '16 at 08:08
  • That path is the default path when using the official Google Protobuf Nuget package. – Tamir Daniely May 15 '16 at 15:13
  • Nuget protobuf3 package by google (3.0.0-beta-2) does not include it. It is a pre-release package with support for version 3. – Davorin Ruševljan May 17 '16 at 08:35
  • I really like this, but I can't get it to work. The ProtobufGenerate task is never run by the build, despite it being set up to do so by the CompileDependsOn. – Max Palmer Aug 04 '17 at 11:51
  • I guess there's something non-standard about your build script. Possibly using a custom targets file or something. AFAIK CompileDependsOn is still the right hook in VS 2017. – Tamir Daniely Jan 24 '18 at 10:39
5

well, that gave me an idea (something about reinventing the wheel)...

  • create simple Makefile.mak, something like
.SUFFIXES : .cs .proto

.proto.cs:
    protogen\protogen.exe -i:$? -o:$@ -t:protogen\csharp.xlst

(obviously, don't forget to replace paths to protogen and csharp.xlst). IMPORTANT - protogen\protogen.exe command starting from TAB character, not 8 spaces

  • If you don't want to specify files needed to be build all the time, you might use something like
.SUFFIXES : .cs .proto

all: mycs1.cs myotherfile.cs

.proto.cs:
    protogen\protogen.exe -i:$? -o:$@ -t:protogen\csharp.xlst
  • in pre-build step to add
cd $(ProjectDir) && "$(DevEnvDir)..\..\vc\bin\nmake" /NOLOGO -c -f Makefile.mak mycs1.cs myotherfile.cs

or, if you have nmake in your path, one can use

cd $(ProjectDir) && nmake /NOLOGO -c -f Makefile.mak mycs1.cs myotherfile.cs
WrightsCS
  • 50,551
  • 22
  • 134
  • 186
Andrey Tch.
  • 91
  • 2
  • 3
  • A plus with this approach is that it's independent of the VS version. A minus (minor) is that a rebuild all of one configuration dirties the build of the others (e.g. rebuilding Debug dirties Release). – Edward Brey Dec 12 '12 at 13:53
2

I have attached a quick and dirty Visual Studio Custom Tool wrapper around ProtoGen.exe to the Google Code page for this issue (http://code.google.com/p/protobuf-net/issues/detail?id=39). This makes adding .proto files to C# projects extremely easy.

See the readme in the attachment for more info.