How I compile the HDF5 Type Provider

I have an older blog post with links to the type provider files for the HDF5 Type Provider for HDF version 1.8.9. Now the time has come that I list links for the version 1.8.14 files.

Solution layout

My Visual Studio 2015 Preview solution for the HDF5 Type Provider has this directory layout:

C:\USERS\RODHERN\HDF5TYPEPROVIDER
└───bin
    └───ChosenConfig
        └───HDF5
            └───1.8.14
                ├───x64
                └───x86

In the main folder (C:\USERS\RODHERN\HDF5TYPEPROVIDER) I have these nine files:

HDF5TypeProvider.fsproj
AssemblyInfo.fs
ProvidedTypes.fsi
ProvidedTypes.fs
HDF5-1814.fsi
HDF5-1814.fs
HelperFunctions.fs
HDF5TypeWrappers.fs
HDF5TypeProvider.fs

The two most important ingredients are the special folders, “x86” and “x64”, and the library import declarations, “HDF5-1814.fs”.

The HDF5 binary DLLs

As mentioned one of the two vital ingredients is the two folders “x86” and “x64” with all the binary HDF5 dynamic link libraries. Ultimately we get the binary libraries from The HDF Group.

When you install a pre-built version of the HDF5 libraries for Windows you get one particular folder, named “bin”, that actually contains the pre-built library files. As I noted in an older blog post, Getting HDF5, the easiest way to get the “bin” folders for both the x86 and x64 compiled versions is to install both, one at a time, and copy the “bin” folder for each one. The version to get this time is obviously HDF5 version 1.8.14.

By the way, you can not use the import declarations in this post for the Itanium platform – the “x64” name refers to the 64 bit versions of the old fashioned Intel processors only.

Just to reiterate, the folder in the above tree view named “x86” is simply the “bin” folder copied after installing the x86 compiled HDF5 libraries, and “x64” is the “bin” folder copied after installing the x64 compiled HDF5 libraries.

From the Copyright Notice and License Terms for HDF5 it seems to me that it would be perfectly legal to distribute the “bin” folders without the executable installers. However I guess we would then need to build the folders from the source. I have almost no idea how to do that. Wouldn’t it be cool though to download the library files with e.g. Chocolatey. Comments to this issue will be most welcome.

Please note that if you use the default Visual Studio configurations the folder for the current configuration will not be named “ChosenConfig” as in the above case. The folder may be named for instance “Debug” or “Release” instead. It is one of my quirks that I prefer to remove the debug and release configurations and settle for just one configuration choice only. The current configuration folder name is not important.

The import declarations

The import declarations themselves and the corresponding signature file are found here:

The F# Type Providers Starter Pack

The type provider sample template files first appeared from Microsoft when introducing the new functionality to Visual Studio (F#). An easier solution became available when Tao Liu published the files as a Type Provider Template (blog post announcement). The files can be found in an up-to-date version as part of the F# Type Provider Starter Pack.

The two newest source files, of the F# Type Provider Starter Pack, are found at these addresses:

The only change that I made to the files is that I replaced “#nowarn “52”” with “#nowarn “25””. I think this makes more sense and I also think the idea must have been all along to disable warning 25 and not 52, but this twist possibly have existed since the early Microsoft preview versions. In any case it won’t make any difference to the compiled type provider so you can just leave the source as is.

The functionality

The provided type functionality – if I can call it so – is sort of spread out into three source files. The idea is that once the type provider is given the file name of the template H5 file it reads the structure from the template file and builds the structure as a tree of properties that can be invoked as shown in the previous blog post. The three files can be found here:

The first file, HelperFunctions, contains read and write functions that the provided types can call.

The second file, HDF5TypeWrappers, defines the wrapper types, HDFFile, HDFGroup, HDFDataSet and HDFAttribute, that will eventually show up as provided types.

Finally, the last file, HDF5TypeProvider, is where TypeProviderAssemblyAttribute and TypeProviderAttribute are applied. The attributes tell Visual Studio that this project is a type provider project. The attributes also let Visual Studio know the namespace name, “Samples.TypeProviders”, and the name of the type, “HDF5File”, to use when a new project references the (compiled) type provider.

The project files

The Visual Studio project file, “HDF5TypeProvider.fsproj”, and the assembly information file, “AssemblyInfo.fs”, are plain files of the kind Visual Studio creates when creating a new blank Visual F# library project.

The result

If you are reading this post it will be no surprise to you that the files must be listed in correct order in “Solution Explorer” in Visual Studio, so let us assume that you already sorted this (see the “Solution layout” section above).

All that is left to do now is to let Visual Studio build the project. The result is an .Net assembly named “HDF5TypeProvider.dll”. This is the assembly that you will reference in the future when you create applications that uses the HDF5 Type Provider.

If you feel there is anything I left out or did not explain properly please let me know in the comments.

Robert Nielsen, 2015-apr-18.

Advertisements

About Robert

This blog to replace the Livespaces one.
This entry was posted in F# and tagged , , , . Bookmark the permalink.

4 Responses to How I compile the HDF5 Type Provider

  1. Pingback: F# Weekly #16, 2015 | Sergey Tihon's Blog

  2. memura says:

    Hello Robert,
    I like what you are doing … have you considered creating a nuget package for this? I think the flexibility your approach brings to including HDF5 in .NET is very useful and is worth posting as a package..
    If you are thinking about this then I am happy to help.
    memura

    • Robert says:

      Hello memura

      A NuGet package might be a good idea. I see a few hurdles in the path to getting there though.

      The most significant hurdle is how to get redistributable binary HDF5 library code. If someone is able to build the binaries from source it seems it can legitimately distributed ( http://www.hdfgroup.org/ftp/HDF5/current/src/unpacked/COPYING ).

      The second hurdle is that I don’t know which files should be included in the NuGet package. To use the type provider in Visual Studio both Visual Studio needs access to the HDF binary libraries (while coding) and the final compiled solution (at run-time) needs access to the HDF binary libraries. Possibly it is a question of distributing the libraries twice? Also is it best to include the compiled type provider assembly or do you normally upload the source files as is?

      Best regards
      Robert Nielsen

  3. Robert says:

    I got a hint on Twitter that maybe the compiled native HDF5 libraries (32 bit and 64 bit ones) are packaged up in this NuGet package https://www.nuget.org/packages/hdf5-v120/ . I will have a look.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s