The Boy or Girl Paradox (conditional probability)

I watched this Youtube clip, TED-Ed’s Frog Riddle Is Wrong, from Presh Talwalkar’s Youtube channel, MindYourDecisions. I enjoy and appreciate the posts on the channel, but in this case I do not completely agree with setup presented, and so I decided to write my assumptions in an even more formal way.

Introducing the paradox

The paradox is a set of two (possibly different) probability problems. The solution to problem two is compared to the solution to problem one to see how similar the solutions seem.

There is a part of the problems that is common to both problems. In both problems we imagine that “Mr. Jones has two children”.

In problem one, (A), we are asked, what is the probability that Mr. Jones has a girl and a boy, given that we know that at least one of the two children is a boy?

In problem two, (B), we are asked, what is the probability that Mr. Jones has a girl and a boy, given that we are told that one of the children is a boy born on a Tuesday?

You may have noticed that I have not copied the premise text exactly as stated in the clip’s riddle. This is a conscious decision on my part. I feel that the fact that the two problems are formulated directly beneath each other might convey interpretational information about problem two, (B), that might be of relevance to the solution.

Solution to problem one (A)

Most people agree on how to solve problem one.

If all we knew was that Mr. Jones has two children, we would say that the four possibly gender combination possibilities, “GG”, “GB”, “BG” and “BB”, was equally probable. In two of the four cases Mr. Jones would have a girl and a boy, so our answer would have been 50 % (= 2/4).

The notation here being that “GG” means both children are girls, “GB” means the elder is a girl and the younger a boy, “BG” means the elder is a boy and the younger a girl and “BB” means both children are boys.

If we already know that at least one of the children is a boy most of us agree to use the (conditional) probability that out of the three possible, and equally probable, outcomes, “GB”, “BG” and “BB”, two of them mean that Mr. Jones has a girl and a boy. So we answer 67 % (= 2/3).

Elaborate solution to problem one (A)

We expect trouble ahead, in problem two, so why not elaborate some more on the example that we can agree on.

Let us denote the gender and birthday of a child with:

• Gt – for a girl born on a Tuesday.
• Go – for a girl born on a day other than a Tuesday.
• Bt – for a boy born on a Tuesday.
• Bo – for a boy born on a day other than a Tuesday.

Any pair of siblings can be described by one of the following sixteen combinations:

[Gt:Gt] 1/196	[Gt:Go] 6/196	[Gt:Bt] 1/196	[Gt:Bo] 6/196
[Go:Gt] 6/196	[Go:Go] 36/196	[Go:Bt] 6/196	[Go:Bo] 36/196
[Bt:Gt] 1/196	[Bt:Go] 6/196	[Bt:Bt] 1/196	[Bt:Bo] 6/196
[Bo:Gt] 6/196	[Bo:Go] 36/196	[Bo:Bt] 6/196	[Bo:Bo] 36/196

I bet you already guessed what my notation is supposed to mean. For instance “[Bt:Go]” means that the first-born child is a boy born on a Tuesday and the younger child is a girl born on a day other than a Tuesday. The (unconditional) probability of “[Bt:Go]” is 6/196 (1/2 * 1/7 * 1/2 * 6/7 = 3 %) as noted in the table.

If we wanted to, we could tally up the number of cases (out of the original 196) with at least one boy (147 cases) and the number of cases with a girl and a boy (98 cases). The answer derived in this way, 67 % (= 98/147), match the answer we already know.

No dark magic so far, so let us see if we can’t stir up a bit of trouble …

Solution to problem two (B) – classical

One way to solve problem two is to look at the above table of the sixteen possible combinations for a pair of siblings, remove the combinations that we know cannot apply to problem two, (B), and check how often you will encounter a boy and girl sibling pair among the combinations that are left.

That solution, presented in the Youtube clip, will have us look at a table not unlike the following, where only combinations satisfying “at least one of the children is a boy that is born on a Tuesday” remain:

[Gt:Gt] –	[Gt:Go] –	[Gt:Bt] 1/27	[Gt:Bo] –
[Go:Gt] –	[Go:Go] –	[Go:Bt] 6/27	[Go:Bo] –
[Bt:Gt] 1/27	[Bt:Go] 6/27	[Bt:Bt] 1/27	[Bt:Bo] 6/27
[Bo:Gt] –	[Bo:Go] –	[Bo:Bt] 6/27	[Bo:Bo] –

The number 27 is just the sum of the number of cases left in the table (i.e. 1 + 6 + 1 + 6 + 1 + 6 + 6).

The solution, when the problem is interpreted this way, becomes that in 14 cases out of 27 cases Mr. Jones has a girl and a boy. The 14 cases are the combinations [Gt:Bt], [Go:Bt], [Bt:Gt] and [Bt:Go], and the rest of the 27 cases are the cases [Bt:Bt], [Bt:Bo] and [Bo:Bt].

Interpretation of problem two (B)

The ‘paradox’ is that when we compare the solution to problem one (A), 67 % (= 2/3 = 98/147), with the (naive or classical) solution to problem two (B), 52 % (= 14/27), we see that the results differ.

There is a quite serious caveat to the interpretation though!

In the way the ‘paradox’ is presented to us it seems to me that the argument goes something like this: “You are in an almost similar situation to problem one, (A); only this time you are told that the boy (in question) was born on a Tuesday”. I imagine that this information is told to me by someone who knows Mr. Jones well. This interpretation leads me to a different calculation and another answer to problem two, (B).

Solution to problem two (B) – interpreted

So someone that knows Mr. Jones well is giving me the information that at least one of the two children is a boy and that the boy is born on a Tuesday. I quite naturally wonder what other possible pieces of information this person might have given me in case the situation had been another. If for instance the boy was born on a Wednesday, what had I been told then?

We can of course imagine all kinds of behaviors with regard to what we will be told. Why not just consider one (more or less arbitrary) strategy that I find reasonable and see what kind of answer we come up with then.

I imagine that the person giving me information about the children does it like this:

• If both children are girls I will be told that “Mr. Jones does not have any boys”.
• If the first-born is a boy I will be told that “Mr. Jones has at least one boy” and I will be told the birthday of the first-born, i.e. Tuesday, Wednesday or what ever day he is born. I will not be told that the boy in question is the first-born.
• If the first-born is a girl and the younger child is a boy I will be told that “Mr. Jones has at least one boy” and I will be told the birthday of the younger child, i.e. Tuesday, Wednesday or what ever day he is born. I will not be told that the boy in question is the younger child.

Let me identify the possible, mutually exclusive, pieces of information with roman numerals:

1. (I) No boys.
2. (II) At least one boy and the day mentioned is a Tuesday.
3. (III) At least one boy but the day mentioned is not a Tuesday.

With the above algorithm in mind we can repeat the above table showing the 16 possible combinations and assign roman numerals indicating the resulting information we would be told in each case:

[Gt:Gt] (I) 1/196	[Gt:Go] (I) 6/196	[Gt:Bt] (II) 1/196	[Gt:Bo] (III) 6/196
[Go:Gt] (I) 6/196	[Go:Go] (I) 36/196	[Go:Bt] (II) 6/196	[Go:Bo] (III) 36/196
[Bt:Gt] (II) 1/196	[Bt:Go] (II) 6/196	[Bt:Bt] (II) 1/196	[Bt:Bo] (II) 6/196
[Bo:Gt] (III) 6/196	[Bo:Go] (III) 36/196	[Bo:Bt] (III) 6/196	[Bo:Bo] (III) 36/196

This time when we do our removal trick we get the table below, where only information option (II) cases are allowed to remain:

[Gt:Gt] (I) –	[Gt:Go] (I) –	[Gt:Bt] (II) 1/21	[Gt:Bo] (III) –
[Go:Gt] (I) –	[Go:Go] (I) –	[Go:Bt] (II) 6/21	[Go:Bo] (III) –
[Bt:Gt] (II) 1/21	[Bt:Go] (II) 6/21	[Bt:Bt] (II) 1/21	[Bt:Bo] (II) 6/21
[Bo:Gt] (III) –	[Bo:Go] (III) –	[Bo:Bt] (III) –	[Bo:Bo] (III) –

As last time around, the number 21, is just the sum of the remaining number of cases.

The solution to problem two, when interpreted this way, then becomes that in 14 cases out of 21 cases Mr. Jones has a girl and a boy. This is indeed the well-known answer, 67 % (= 2/3 = 14/21), that we recognize from problem one, (A).

The morale of the story

The difference between the two candidate answers stems from the “Bo;Bt” cases. We might phrase the observation as a question (or two):

“What would we be told if Mr. Jones had two boys and they were born on different days of the week? Which boy or which day of the week would be assigned more importance?”.

Sure, if, for instance, you are willing to accept that Tuesdays somehow, in the context of the paradox, are particular important days, then yes possibly the answer might be 52 % (= 14/27). On the other hand, the answer might as well be 67 % (= 14/21).

To me the interesting statistical tidbit to take away from this paradox is that, if you regard the information about the day of the week as pointless then that information indeed turns out to be insignificant, whereas if you believe you are given important information about the birthday then that bit of information is quite significant. The good news is that you will always be right!

Robert Nielsen, 2016-jun-03.

How I compile the HDF5 Type Provider

I have an older blog post with links to the type provider files for the HDF5 Type Provider for HDF version 1.8.9. Now the time has come that I list links for the version 1.8.14 files.

Solution layout

My Visual Studio 2015 Preview solution for the HDF5 Type Provider has this directory layout:

C:\USERS\RODHERN\HDF5TYPEPROVIDER
└───bin
    └───ChosenConfig
        └───HDF5
            └───1.8.14
                ├───x64
                └───x86

In the main folder (C:\USERS\RODHERN\HDF5TYPEPROVIDER) I have these nine files:

HDF5TypeProvider.fsproj
AssemblyInfo.fs
ProvidedTypes.fsi
ProvidedTypes.fs
HDF5-1814.fsi
HDF5-1814.fs
HelperFunctions.fs
HDF5TypeWrappers.fs
HDF5TypeProvider.fs

The two most important ingredients are the special folders, “x86” and “x64”, and the library import declarations, “HDF5-1814.fs”.

The HDF5 binary DLLs

As mentioned one of the two vital ingredients is the two folders “x86” and “x64” with all the binary HDF5 dynamic link libraries. Ultimately we get the binary libraries from The HDF Group.

When you install a pre-built version of the HDF5 libraries for Windows you get one particular folder, named “bin”, that actually contains the pre-built library files. As I noted in an older blog post, Getting HDF5, the easiest way to get the “bin” folders for both the x86 and x64 compiled versions is to install both, one at a time, and copy the “bin” folder for each one. The version to get this time is obviously HDF5 version 1.8.14.

By the way, you can not use the import declarations in this post for the Itanium platform – the “x64” name refers to the 64 bit versions of the old fashioned Intel processors only.

Just to reiterate, the folder in the above tree view named “x86” is simply the “bin” folder copied after installing the x86 compiled HDF5 libraries, and “x64” is the “bin” folder copied after installing the x64 compiled HDF5 libraries.

From the Copyright Notice and License Terms for HDF5 it seems to me that it would be perfectly legal to distribute the “bin” folders without the executable installers. However I guess we would then need to build the folders from the source. I have almost no idea how to do that. Wouldn’t it be cool though to download the library files with e.g. Chocolatey. Comments to this issue will be most welcome.

Please note that if you use the default Visual Studio configurations the folder for the current configuration will not be named “ChosenConfig” as in the above case. The folder may be named for instance “Debug” or “Release” instead. It is one of my quirks that I prefer to remove the debug and release configurations and settle for just one configuration choice only. The current configuration folder name is not important.

The import declarations

The import declarations themselves and the corresponding signature file are found here:

• HDF5-1814.fs
• HDF5-1814.fsi

The F# Type Providers Starter Pack

The type provider sample template files first appeared from Microsoft when introducing the new functionality to Visual Studio (F#). An easier solution became available when Tao Liu published the files as a Type Provider Template (blog post announcement). The files can be found in an up-to-date version as part of the F# Type Provider Starter Pack.

The two newest source files, of the F# Type Provider Starter Pack, are found at these addresses:

• ProvidedTypes.fsi
• ProvidedTypes.fs

The only change that I made to the files is that I replaced “#nowarn “52”” with “#nowarn “25””. I think this makes more sense and I also think the idea must have been all along to disable warning 25 and not 52, but this twist possibly have existed since the early Microsoft preview versions. In any case it won’t make any difference to the compiled type provider so you can just leave the source as is.

The functionality

The provided type functionality – if I can call it so – is sort of spread out into three source files. The idea is that once the type provider is given the file name of the template H5 file it reads the structure from the template file and builds the structure as a tree of properties that can be invoked as shown in the previous blog post. The three files can be found here:

• HelperFunctions.fs
• HDF5TypeWrappers.fs
• HDF5TypeProvider.fs

The first file, HelperFunctions, contains read and write functions that the provided types can call.

The second file, HDF5TypeWrappers, defines the wrapper types, HDFFile, HDFGroup, HDFDataSet and HDFAttribute, that will eventually show up as provided types.

Finally, the last file, HDF5TypeProvider, is where TypeProviderAssemblyAttribute and TypeProviderAttribute are applied. The attributes tell Visual Studio that this project is a type provider project. The attributes also let Visual Studio know the namespace name, “Samples.TypeProviders”, and the name of the type, “HDF5File”, to use when a new project references the (compiled) type provider.

The project files

The Visual Studio project file, “HDF5TypeProvider.fsproj”, and the assembly information file, “AssemblyInfo.fs”, are plain files of the kind Visual Studio creates when creating a new blank Visual F# library project.

The result

If you are reading this post it will be no surprise to you that the files must be listed in correct order in “Solution Explorer” in Visual Studio, so let us assume that you already sorted this (see the “Solution layout” section above).

All that is left to do now is to let Visual Studio build the project. The result is an .Net assembly named “HDF5TypeProvider.dll”. This is the assembly that you will reference in the future when you create applications that uses the HDF5 Type Provider.

If you feel there is anything I left out or did not explain properly please let me know in the comments.

Robert Nielsen, 2015-apr-18.

How to use the HDF5 Type Provider

A friendly reader left a comment reminding me that I never really told how the HDF5 Type Provider is used in a Visual Studio F# code project.

In this post – sample code

It has been a while since I last took a look at the HDF5 type provider example – or any F# type provider for that matter. Looking at the example again I must admit that it helps a lot to have a few lines of sample code showing how to invoke the type provider.

I will copy-paste a few sample code lines showing what the HDF5 Type Provider can do.

In a future blog post I might spell out how to get hold of the HDF5 library and type provider files (version 1.8.14). The older explanations (version 1.8.9) can be found in blog posts Getting HDF5 and Type Provider Template .

Reference the HDF5 Type Provider

A natural first step is to create a new F# project. I am using one of the Visual Studio 2015 Previews. In this case I am creating a fresh Visual F# Console Application called “FSHDFConsoleApp”.

The second step is to let Visual Studio know that I want to use the HDF5TypeProvider assembly. This is done by adding a reference to “HDF5TypeProvider.dll” in exactly the same way as when referencing any other .Net assembly from within the newly created FSHDFConsoleApp project.

The first time you reference a new type provider assembly Visual Studio will warn that the type provider executes custom code and Visual Studio asks if you want to enable the type provider. Note that once you answer to enable the type provider Visual Studio will remember your choice. The Visual Studio list of trusted type provider assemblies can be accessed from “Options” > “F# Tools” > “Type Providers”.

It is now time to actually reference the type provider content from the FSHDFConsoleApp project source code (in this case “Program.fs”). To do this I need three pieces of information:

1. I need to know the name of the relevant namespace in order to get to the provided types. The namespace name is hardcoded into the HDF5 Type Provider. The namespace is “Samples.TypeProviders”.
2. I need the name of the provided type definition and the syntax used to obtain ‘new types’ on the fly. The provided type definition is called “HDF5File”. The syntax used is shown below.
3. The type provider needs to know the layout of the HDF5 file that I want to create a ‘new type’ for. I tell the type provider the layout by letting it read a template file. A template file is simply an HDF5 file that has the same layout as the HDF5 file(s) that my executable program eventually will be working with. The numerical data in the template may be bogus but the structure, such as group and variable names, have to be pretty much the exact same as in the data files that will ultimately be processed by the program. The type provider gets to know about the template file(s) through the special syntax shown below.

Load template files

// Open the namespace defined in the type provider;
// Visual Studio already checked the "Root Namespace"
// in the type provider assembly.

open Samples.TypeProviders

 
// Each ‘type’ is given its own template, so if you use more than one
// template you should create a separate ‘type’ for each used template.

let  [< Literal >]
     sometemplatefilename = @"SomeTemplate.H5"
type SomeHDFFile = HDF5File<sometemplatefilename>

let  [< Literal >]
     othertemplatefilename = @"OtherTemplate.H5"
type OtherHDFFile = HDF5File<othertemplatefilename>
                                                                         

 

It should come as no surprise that the template file(s) must be accessible or Visual Studio can’t get the type provider code to run. When the type provider code fails (raises an exception) an error is shown in the Visual Studio GUI.

Notice that the error message, at least in this case, helps hint at the probable cause.

Getting an HDF5 file wrapper

The above class, SomeHDFFile, is created based on the layout of the template file, “SomeTemplate.H5“. The purpose of the program code of the FSHDFConsoleApp project is not to manipulate the template file, but to read and write data from ‘future’ data files with similar layout. Often we won’t know the paths and names of the files ultimately processed, but in this case, for the sake of simplicity, let us assume that we can hardcode the ‘future’ file name, “MyDoubles.H5”.

To get a file wrapper object I call the constructor of SomeHDFFile passing the (path and) name of the ‘future’ (actual) HDF5 data file.

Instantiate wrapper object

// At some point, when the code is actually executing, we need a
// path to what ever file we are actually going to read.
// Let us pretend it is this one.

let getruntimefilename () = @"MyDoubles.H5"

// Get file used at run-time (wrapper)

let runtimefile = new SomeHDFFile (getruntimefilename())
                                                                           

 

The resulting variable, “runtimefile”, behaves somewhat like a handle for the HDF5 data file “MyDoubles.H5”. The handle can be thought of as a wrapper in the sense that the more important actions done to the wrapper are actually done to the underlying HDF5 data file.

Reading a matrix value

The only genuine advantage to a type provider, of the HDF5 Type Provider kind, is the Intellisense that is provided. If you don’t want the type provider intellisense it is hardly worth jumping through all the type provider hoops – just use the interface files themselves.

Let us see what intellisense shows up for runtimefile:

The main property is called “Root”. The top level entry of the HDF5 data file is always presented as a root group called “Root”. This is a decision I made in the design of the HDF5 Type Provider. The root group is not actually in the HDF5 data file, it is something I have put in to give the same feel to data groups and the data file root entry. So let’s see what’s in Root:

In this case there is a group named “G”. The group was created as a three field structure in Octave. The reason that “G” is described as a one data set group is that Octave does a trick when saving variables to HDF file format. Octave bundles up the numeric data and meta data in an HDF group. The actual data go into a subgroup or data set called “value”. The meta data bundling is not technically necessary as Octave will read HDF files without Octave meta data just fine. Seen from an F# point of view Octave tugs away the interesting data in the “value” property.

The three fields of the structure is “DynamicGroup”, “M” and “Pi”. All three of these can be found inside the “value” property.

Variable “M” is a floating point matrix – let’s say that it is the one we are interested in. As already mentioned, when Octave saved the Matrix, M, Octave did a trick. The actual matrix data was put in a  data set called “value” and saved alongside some Octave meta data (string data “type” and attribute “OCTAVE_NEW_FORMAT”). In this case the Octave HDF save trick does make a bit of noise to the Visual Studio Intellisense. What we think of as matrix M is somewhat confusingly described as a “Group” in the type provider type ‘documentation’. We will, nonetheless, soldier on and step inside M:

Ah, finally, a matrix (i.e. a two-dimensional array) is in our sight. Now that we are here, actually the journey probably wasn’t all that hard. After all, we can just keep invoking the Visual Studio Intellisense to explore the template file(s).

In the current implementation of the HDF5 Type Provider (1.8.14) the data set wrapper is the same regardless of whether the data set is a scalar value, a vector, a matrix, an n-dimensional array or for that matter some entirely fifth kind of data set. The downside to this is that it is up to us to select the appropriate method to load the data set in question. In this case the proper choice is pretty obvious though; ReadMatrix will do the trick!

Our newest code line, runtimefile.Root.G.value.M.value.ReadMatrix (), can now be combined with the above code segments. The last part of the code got to look like this:

Load matrix from file

// Load matrix valued variable

printfn "Load M:"
try
  runtimefile.Root.G.value.M.value.ReadMatrix ()
  |> printfn "%A"
  with
  | e -> printfn "%s" e.Message
                                                                           

 

Just like Visual Studio needs Platform Invoke access to the binary HDF5 libraries to be able to read the template files the executing program, FSHDFConsoleApp.exe, also needs to Platform Invoke into the HDF5 libraries at runtime.

In my case the above code produces this result:

Seeing this post already growing quite longwinded let me end it with an example showing how to write over an existing matrix with new cell values.

Writing values and see the result

The provided functionality in the HDF5 Type Provider will allow you to write over an existing matrix only if the size is unchanged. For this reason I keep M a 3 times 5 array in this example.

Replace old cell values

// Create a new matrix and save it to file

let M =
  fun j i
   -> float (1 + i) + float (1 + j) / 10.
  |> Array2D.init 3 5

do runtimefile.Root.G.value.M.value.WriteMatrix M
                                                                           

 

.Net arrays are (usually) zero-based, so adding one to i and j will make the indices appear one-based. In this example the array M have index j run through three ‘columns’ and index i run through five ‘rows’. The cell value at the i’th row and the j’th column of the matrix is technically the array entry “M.[j-1,i-1]“. For instance the cell value at the fourth row and second column of the matrix becomes 4.2.

The result can be viewed in HDFView.

By default HDFView presents the matrix in array style (sideways), in the same way as the default F# print command (printfn “%A”) does. In the HDFView User Options you get a choice between presenting arrays as “0-based” or “1-based”; the “1-based” option was chosen for the above screenshot. As noted in blog post F# to Octave the matrix, M, is at the same time considered a 3 times 5 array and a 5 x 3 matrix.

As a last point let me check that Octave is able to read back the HDF5 data file, “MyDoubles.H5”, that was just modified.

We see that the updated matrix has found its way back to Octave.

Robert Nielsen, 2015-mar-15.

F# script reminder

This week I ran into a problem that I last encountered a year or two ago. The problem is how to set up the Windows shell “Run with F# Interactive” right-click menu option for F# script files.

Invoking FSI from the Windows Explorer context menu

When you install Visual Studio you get a Windows Explorer context menu item that will run F# scripts using F–Sharp Interactive (fsi.exe a.k.a. FSI) from the F# Tools bundle.

The problem the last time was that even though you could debug to your heart’s content from within Visual Studio it wasn’t that easy to debug scripts run directly from the Windows Explorer context menu.

By default the Visual Studio right-click “Run with F# Interactive” option launches FSI in its own window which will disappear when the script stops. This also applies when the script halts due to an unhandled exception. This meant that I couldn’t get information about the exception that was raised by the script because the console output, and indeed the entire FSI window, disappeared almost at the same instance that the exception occurred. I didn’t get any debug messages and nothing went to the trace listeners either. The problem was quickly solved because I got a lot of help online almost immediately when asking for help over at FPish.

This time around the problem arose in a different variation – I wanted to add the right-click option from ‘scratch’. This situation may happen if you install only “Microsoft Visual F# 3.1.2” – maybe you use a basic editor and skip the Visual Studio installations. In my case it happened because I removed an old decommissioned Visual Studio trial or preview installation that was somehow tied to an old F# installation.

My setup at this stage had a Windows Explorer context menu looking like this:

To set up a file explorer right-click menu item for F–Sharp Interactive I turned to the answers given to me last time. The following three tutorial-like blog posts contain all the information needed to do the job:

• Running F# Interactive from Windows context menu
• Scripting in F#
• Add Application Shortcut to Windows Explorer’s Context Menu

The first one is the important one, it explains how to set up cmd.exe to launch fsi.exe to execute a script file, the second gives a good general account of scripting in F# and the third shows you how to find your way in regedit.

My end result looks like this:

I decided to take only the most important points from the posts and show how to add the “Run with F# Interactive” option to your new F# installation. Below are the five steps I performed.

Recipe

• Open RegEdit

Launch the graphical Windows registry editor. To do this you can press the Windows key plus “R” for run, type in “regedt32” in the run box and press enter. On newer Windows versions this will launch C:\Windows\regedit.exe .

• Navigate to “HKEY_CLASSES_ROOT” and register .fsx file type

Navigate to the “HKEY_CLASSES_ROOT” item. In my registry this is the first node shown under “Computer” (the default common local root).

Search (Ctrl+F) for “.fsx” to check if you already have an association for the F# script files. If you do you should edit the existing entry instead of creating a new one, otherwise the result might not be what you want it to be.

To associate .fsx files with a new (or existing) file type group all we have to do is to create a key and value pair under “HKEY_CLASSES_ROOT”:

The key (node) is called “.fsx” and the (default) data value is a name that we make up for the occasion. As you can see from the above image my file type group is called “f_sharp_script”.

• Create batch (.bat) file

The idea is to write a batch file that will launch FSI.exe. The batch file will have just this one command as its only line. We will then have the Command Processor run the batch command.

We can create the batch file where ever we want but due to my laziness I just put mine in the %appdata% root. You can quickly open up the %appdata% root folder simply by typing “%appdata%” in the run box (Windows+R).

Create a brand new batch file, i.e. a text file with file name extension “bat”. I named mine “runcmdfsi.bat”. Edit the new batch file in Notepad or similar tool so that the batch file invokes the F-Sharp Interactive executable. The command should consist of three parts. The first part is the name and path of the executable, preferably in quotes. Note that it isn’t necessary to add the path to the Path environment variable. The second part is a selection of all the command line arguments and flags you like to use. The last part is a reference to the script file, which is simply “%1” (without the quotes).

My batch file contents (it is all one line) is this:

“C:\Program Files (x86)\Microsoft SDKs\F#\3.1\Framework\v4.0\FSI.exe”
–optimize –tailcalls -d:DEBUG -d:TRACE –quiet –exec %1

• Reference the batch (.bat) file from the registry

To reference the batch file we create a particular “command” key in the registry. We will create the “command” key (node) inside an other newly created node inside yet an other newly created node inside … okay let’s just note the path to the value already:
HKEY_CLASSES_ROOT\f_sharp_script\shell\runcmdfsi\command.

Here “f_sharp_script” is the name of the file type group. I made up the name earlier (in step 2 above).  The second to last part, “runcmdfsi”, can be named just about anything, so I picked a name that helps remind me what the command does.

The path with the new command key and data value is seen here:

The text of the command is:

“c:\windows\system32\cmd.exe” /Q /K %%appdata%%\runcmdfsi.bat “%1”

While we are at it let us name the command. We do this by assigning a (default) string data value to the “runcmdfsi” key. The name will show up when we right-click an F# script file in Windows Explorer. The name may be a short description of the command. I think “Run with F# Interactive” is a suitable choice:

• Check that everything works

We better try out the new command to be sure it works as intended. The new FSI action doesn’t need to be the default action for script files – in fact ‘Edit’ may make more sense as the default action. To try out the new command we right-click a script file (.fsx) and click “Run with F# Interactive”.

The test script obviously doesn’t need to be anything special – certainly my script contains nothing profound:

#light
open System
open System.Diagnostics

do printfn “The script is executing!”
do Trace.WriteLine(“Output to trace listeners.”);
do failwith ” … and this makes the script fail.”

If you don’t have a suitable script file at hand already just create a new one somewhere. It is unimportant how you name the the script and where you put it – you can remove it in a minute anyway.

Executing the script via the menu item opens a command window and leaves it open:

I guess the only thing left to do now is to celebrate the powerful .Net scripting that we have available even if we install only one of the more rudimentary F# packages.

Robert Nielsen, 2014-nov-15.

Xamarin Studio 5 Editor

Initial impression

I opened up Xamarin Studio 5, let it update to the newest version and created a new F# project.

This was a pleasant surprise. Even though I leave everything on the default choice I get a well known project structure with a main solution file (.sln file), a project folder, a project file (.fsproj file) and some source code files. It gets better still, the files by default are UTF‑8 encoded text files saved with byte order mark – even the solution file which is in fact a Microsoft Visual Studio Solution File (for VS2012).

I guess that means I can work on the same set of files in Xamarin Studio, Visual Studio, EditPlus, SharpDevelop or what ever tool I fancy for the day. To check that integration indeed is easy I went to Visual Studio Online and created a Git repository. If you do not already have a Visual Studio Online account you can sign up for one for free here:
http://www.visualstudio.com/products/visual-studio-online-Basic-vs .

I added “alternate credentials” to my Visual Studio Online account. The alternate credentials consist of a user name and alternate password that can be used by Git. That way you can keep your Git access and your Live Id separate. By the way, if you want to use a Visual Studio Online Git repository and access it via Git or Xamarin Studio from ‘the outside’ alternate credentials are compulsory.

The setup works. I can push changes made in the Xamarin Studio editor to the online repository and see the changes from my personal Visual Studio page online.

Another nice surprise is that the default setup of windows in Xamarin Studio – again without any tweaking needed – includes a solution explorer window, an editor window and an F# Interactive (FSI) window, all open and ready to fire away. Getting right to work with FSI running is a nice bonus.

Unicode problem (version 5.5.0)

On the more disappointing side of things there seemed to be a problem with the editor character set.

When I sent commands to FSI unicode characters without an ANSI sibling would be displayed as garbage. So I went to the menu and selected ‘courier new’ as the font for the FSI window. That didn’t work so I changed it back. That certainly didn’t work either. Now I got an FSI window looking like this:

Reinstalling Xamarin Studio did not help. Quite disappointing when you take into consideration that this is supposed to be the fifth major version of the Xamarin Studio editor.

Then in a stroke of luck it just happened that before I got to publish this post another update to Xamarin Studio was ready for download by the built-in Xamarin Studio Updater.

Problem solved (version 5.5.2)

Not much explanation is needed; here is a screenshot of FSI in Xamarin Studio 5.5.2:

Crisis averted!

Robert Nielsen, 2014-oct-24.

Xamarin Studio 5

In January we got a new free editor for F# when Microsoft announced that the new “Visual F# Tools 3.1.1” could be used with “Microsoft Visual Studio Express 2013 for Windows Desktop”. The newest “Visual F# Tools” is version 3.1.2.

Midyear Xamarin launched an F# t-shirt event (http://blog.xamarin.com/run-a-f-sharp-app-get-a-f-sharp-shirt/). The idea was to encourage trying out Xamarin Studio (or the Visual Studio integration) for free.

There is a bit of naming confusion going on at the Xamarin web site. Just look at the main download page, the title says “Xamarin Platfom”, the download button says “Xamarin for Windows” (in my case) and the file I get is called “Xamarin Studio”. On top of that I can’t run anything until I get “Xamarin Gtk#”.

I tried to install Xamarin Studio 5 to two different environments, and I tried running the installation with and without Visual Studio installed. In each case the installation failed with some error message or other.

Last month Christian Jacobsen posted a solution to the Xamarin Studio installation problem. The trick apparently is to install this quite old F# Tools 3.0. You know what? It worked! Thanks Christian.

For what it is worth, the software that I installed (in Hyper-V) was:

• Microsoft Windows 7 Professional 64 bit (with service pack and updates)
• Chocolatey ver. 0.9.8.27
• F# Tools ver. 3.0
• F# Tools ver. 3.1.2
• Xamarin Gtk# for .Net ver. 2.12.25
• Xamarin Studio ver. 5.0.1

I am not trying to imply that all this software is necessary to run Xamarin Studio, and I am not suggesting that the mentioned versions are better than other versions either. If you already downloaded the Xamarin Studio installation package at some point you probably don’t need to do it again – once installed, when I opened up Xamarin Studio I was offered to update Xamarin Studio to a newer version (ver 5.5.0) by the built-in Xamarin Studio Updater which worked just fine.

A little more than a week ago, Monday October the 6th, Visual Studio 14 CTP 4 was released. Unfortunately I can’t install it. The installer, whether I use the downloaded iso file or the web installer, fails with a message that it can’t install Microsoft .NET Framework 4.5.3 Preview. I get the same error in Windows 7 (Hyper-V) as well as Windows 8 (native boot).

Robert Nielsen, 2014-oct-15.

Monad bind operator for function composition, part 2 of 2

After seeing this clip on YouTube I felt inspired to add a few words and illustrations along the lines of Monads and function composition. This is the second of two posts – and it is time for the Monad to appear.

Recapitulation

In the last post we left off with a figure like this:

The green A, B, C and D denote arbitrary types, and the red f, g and h denote arbitrary functions that map from one type to the next. To avoid dealing with arbitrary types and functions in the example we will reuse the definitions from the first example:

Reused definitions

// Arbitrary types example
type A = int       
type B = float     
type C = DateTime  
type D = string    
                                                           
// Arbitrary functions example
let f (noWeeks: A): B = float (7 * noWeeks)
let g (noDays: B): C = today.AddDays noDays
let h (date: C): D = date.ToShortDateString ()

The lower half of the figure depicts four additional types, R, S, T and U. The lower half also depicts some functions that we won’t worry about yet.

The problem

Let us imagine two functions, f0 and g0, with these signatures:

val f0 : A -> S    
val g0 : B -> T    

If it just happens that the type S is identical to the type B, as was the case in the very first example in the previous post, then of course function composition is straight forward.

Types already lined up

let f0g0 x = g0 (f0 (x))  

If the types S and B are not the same we may need a dedicated translator function. The translator function in the previous post was called composeSB. If you are willing to write a translator function every time you need to combine two functions then that will certainly  work.

Values explicitly converted

let f0g0 x = composeSB(f0 x, g0)   

As you probably guessed already – this post would never have gotten this long if we did not contemplate to suggest a third solution to the problem. So let us assume that neither of the two mentioned solutions are quite adequate.

The third solution

What we want to do is to introduce some kind of relation between types S and B such that we don’t have to make the function output of f0 match the input type of g0; we just need to make it match with a related type.

Specifically we introduce a relation that guarantees every type a type mate. The actual definitions of types R, S, T and U look like this:

type R = M<A>
type S = M<B>
type T = M<C>
type U = M<D>

The full example can be found here:
MonadBind3of3.fsx

Well, what good is it that instead of being forced to match to type B we now need to match to type S? Let us not keep secrets, it is a heck of a lot less generic than the general case (the translator functions). It is, however, not quite as useless as it may first appear. The important point is that there is a good deal of leeway in the way we design M<_>, the mate relation between on one hand types A, B, C, D and on the other hand R, S, T, U.

The Monad

The idea is to have the types (B and S) resemble each other in a predictable way. Basically we create a new kind of type, a type that is used as a container. The type S (i.e. M<B>) will be a container to type B and type T (i.e. M<C>) will be container to type C.

The type M<_> is called the Monadic Type. A Monadic Type is only called a Monad if it has associated functionality to deal with functions with these kinds of signatures:

val Mf: A -> S  // i.e. A -> M<B>
val Mg: B -> T  // i.e. B -> M<C>
val Mh: C -> U  // i.e. C -> M<D>

The mentioned special Monad functionality primarily takes the form of a (builder) class instance method called Bind.

Conventions for Monads

I need to make two comments in respect to the defined Monad, M<_>.

I mentioned that the new kind of type, M<_>, is a container type. There is absolutely no requirement that M<_> must act as a container type, the only requirement is that M<T> can be considered a type mate to type T. The reason M<T> is usually a container type is because it is among the easiest constructions you can create. A container type M<T> typically contains a value of type T and just about any kind of additional information. Often you can avoid storing state information outside of the functions invoked by tagging on the needed state information to the container type (M<_>).

The syntax in F# to deal with computation expressions requires methods such as Bind to be instance methods. I have not yet noticed anyone actually utilizing this. Probably it is easier to figure out what your computation expressions do if you stick to static methods for your major functionality. Also the description of computation expressions hardly mentions Monad laws, which means that if you do not want to you do not have to care about stuff like associativeness of your computation expression elements.

In other words, there are some conventions that Monads are supposed to adhere to, but it is voluntary to do so. After all it is your computation expressions and hence the decisions are yours to make.

Without really explaining what I want from my Monad I pulled one out of the hat. Allow me to make yet two detours that strictly speaking has nothing to do with Monads – pipe operators and function packaging.

Pipe operators

Imagine that I have functions f, g and h that I want to perform one after the other. This means that I will pipe an argument, x, through f, g and h. We did this explicitly in the first post:

let fghExplicit = fun x -> h (g (f (x)))         

There is obviously a lot of possible solutions and one is to use a pipe operator. There is already a pipe operator named (|>) in F#. Despite the fact that (|>) is already built into the language I will define the operator under a new name to show there is no trick to it:

Plain old pipe operator

let pipe1 x f = f x                              

It is easy to use the pipe operator to get the result. I get this result:

let result1 =
    pipe1 (pipe1 (pipe1 x0 f) g) h               

val result1 : D = "2014-03-02"

Next assume that I want to pipe not only the argument through the functions but I want to also pipe along a message string that allows me to see the intermediate results. I decide to pass the original argument and message string as a tuple. The new operator looks like this:

Special purpose pipe operator

let pipe2 (x, s) f =
    let fx = f x
    let msg = sprintf "fun(%s)=%A" s fx          
    fx, msg

The result comes out the other end as a tuple, again with the main argument in the first part and the message string in the second part. I get this result:

let result2, msg2 =
    pipe2 (pipe2 (pipe2 (x0, msg0) f) g) h                   

val result2 : D = "2014-03-02"
val msg2 : string = "fun(fun(fun(2)=14.0)=2014-03-02 20:05:12)=\"2014-03-02\""

The full pipe operator examples can be found here:
MonadBind4of3.fsx

Packaged functions

I mentioned a method called Bind. In my example the (static) bind method looks like this:

Static bind method

member StaticBind (mt: M<'a>) (fn: 'a -> M<'b>) =
  let result = fn mt.Value
  let newmessage = sprintf ("Passing value %+A") mt.Value
  let logmessages = result.LogMessages @ (newmessage :: mt.LogMessages)   
  { Value= result.Value; LogMessages= logmessages }

The definition itself is not important. Two things are worth noting though:

1. StaticBind essentially passes a value to a function while at the same time keeping track of some kind of message log.
2. StaticBind works with functions that have signatures of the form A –> M<B> .

Number 1 tells us that the purpose of StaticBind resembles the purpose of pipe2 . Number 2 tells us that StaticBind is designed to handle the kind of functions that we referred to as Mf, Mg and Mh earlier on in this post.

With the help of StaticBind and one other very simple method, StaticReturn defined below, we can turn ordinary functions f, g and h into corresponding functions pf, pg and ph that do the same job as f, g and h and at the same time uses the StaticBind pipe functionality to pass along and maintain a message log.

Function mates

member StaticReturn v = { Value= v; LogMessages= [] }      

let ret x = M<_>.StaticReturn x
let package fn mt = M<_>.StaticBind mt (fn >> ret)         

let pf: R -> S = package f
let pg: S -> T = package g
let ph: T -> U = package h

I get results like this:

let fghOne: R -> U = compose(compose(pf, pg), ph)
let fghTwo: R -> U = compose(pf, compose(pg, ph))          

let fghOneResult = fghOne (ret myInputParam)
let fghTwoResult = fghTwo (ret myInputParam)

val fghOneResult : U =
  {Value = "2014-03-02";
   LogMessages =
    ["Passing value 2014-03-02 20:05:12";
     "Passing value 14.0";
     "Passing value 2"];}
val fghTwoResult : U =
  {Value = "2014-03-02";
   LogMessages =
    ["Passing value 2014-03-02 20:05:12";
     "Passing value 14.0";
     "Passing value 2"];}

The full example is at the already mentioned link:
MonadBind3of3.fsx

I imagine this approach a bit like putting all the arguments – the input parameters and function results – in box shaped packages. The boxes are roughly similar on the outside even though the inside argument and filler stuff differs from package to package. As we all know box shaped packages are delightfully uncomplicated to pipe along from station to station. The original functions do their usual work at each station along the way and the Monad Bind provides the piping in between.

So, what did our Bind function do?

Finally look at the definition of the static bind method to figure out what the shown StaticBind example actually does. It is so obvious that you might miss it at first glance, the primary task performed is invoking fn. Besides applying fn to the value held by mt StaticBind will maintain a list of log messages. In the example there is theoretically three sources of log messages.

1. The newmessage is constructed on the spot, at the time when both function, fn, and input parameter, mt, are available. I put the newmessage in the middle of the log messages.
2. Further down the list of log messages we have the older messages. The older messages are simply the list of log messages that comes with the input parameter, mt.
3. Finally if log messages are created by fn, which returns a M<_> type value, I suppose those messages are the newest of the lot and I put them at the beginning of the list.

What this is supposed to show is that in general Monads are used to perform your usual (piped) function calls but in circumstances where you want to do something more besides the clean function composition. The Monads are used to add secondary functionality to your existing functionality without braking your existing building blocks.

The reason we don’t just use the packaged functions, pf, pg and ph, is that F# handles computation expression syntax which is special syntax to deal with these kind of examples. In fact the main example may be written like this using computation expression syntax:

Computation expression

let result =
    builder {
              let! a = Mx    
              let! b = Mf a
              let! c = Mg b
              let! d = Mh c
              return d
            }

A screenshot or code quote won’t show you the real benefit of the syntax. The main benefit, to my mind, is that you get more helpful Intellisense. When I hover with the cursor over a, b, c and d I am told that the applicable types are A, B, C and D, as opposed to R, S, T and U. The computation expression syntax makes sure not to flaunt the fact that we are working with M<_> type values, which might make it easier to overlook the Monad functionality and concentrate on the original building blocks.

Robert Nielsen, 2014-feb-16.

Monad bind operator for function composition, part 1 of 2

I saw this clip on YouTube where Brian Beckman tells how we can think of Monads and emphasize that it is all to do with combining several small functions to build the functionality you want.

If you are confused about Monads and you have an hour to spend to watch the talk, and haven’t seen it already, then by all means go see it now. Brian Beckman states that the way to control complexity is to create the complexity by combining smaller simpler building blocks. The way Brian Beckman combines smaller building blocks is by the use of Monads. After watching the clip I felt inspired to add a few illustrations along the lines of the topic of the clip.

There is a risk that this post is going to be fairly long. To avoid making it way too long I have chosen to split it up into two parts. In the first part I will look at the usual way to combine two, three or more functions into one composite function by ordinary function composition. The Monad will appear in the second part.

Function composition

The classic function composition is the basis for combining functions. So let us begin by looking at a very simple example, where I combine three functions, f, g and h, into one calculation.

For this example I will define some specific types to be used by f, g and h.

Types B and C

type A = int
type S = float
type B = S
type T = DateTime
type C = T
type U = string

Next I will give definitions to functions f, g and h. The definitions themselves are unimportant.

Functions f, g and h

let today = DateTime.Now
let f (noWeeks: A): S = float (7 * noWeeks)
let g (noDays: B): T = today.AddDays noDays
let h (date: C): U = date.ToShortDateString ()

The combined function, when applying f, g and h in sequence, gives a text with the date a certain number of weeks from now; for instance two weeks from now (noWeeks = 2).

One way to combine the three functions is to do it explicitly:

Explicit composition

let fghExplicit = fun x -> h (g (f (x)))

Another way, that seem to have more reusability potential, is to define a generic way to compose functions:

Two at a time

let compose (fun1, fun2) = fun arg -> fun2 (fun1 (arg))
let fghOne = compose(compose(f,g),h)
let fghTwo = compose(f,compose(g,h))

The code for this example can be found here:
MonadBind1of3.fsx

The reason this example turns out this easy is because the type of result of f is the same as the type of argument for g, and the type of result of g is the same as the type of argument for h. This is the case because S equals B and T equals C.

But what if the types do not line up?

If the types do not line up, that is if S is not the same as B, we need some way of transforming the result of one function so that it can be input into the next function.

For this example I have chosen some new types.

Types A, S, B, T, C and U

type A = int
type S = int
type B = float
type T = DateTime
type C = int * int * int
type U = string

Also the functions, f, g and h, are modified.

Functions f, g and h

let f (noWeeks: A): S = 7 * noWeeks
let g (noDays: B): T = today.AddDays noDays
let h ((yyyy, mm, dd): C): U = sprintf "%04d-%02d-%02d" yyyy mm dd

The combined function when applying f, g and h in sequence will, as in the first example, give a text with the date for instance two weeks from now (noWeeks = 2).

To combine f and g I write a helper function, composeSB, that knows how to transform a result of type S to an argument of type B. To combine g and h I write a helper function, composeTC, that knows how to transform a result of type T to an argument of type C.

The helper functions look like this:

Transform S to B

let composeSB: S * (B -> 'T) -> 'T =
    fun (x, fn) -> fn (float x)                  

Transform T to C

let composeTC: T * (C -> 'U) -> 'U =
    fun (x, fn) -> fn (x.Year, x.Month, x.Day)   

We notice that the helper functions are specific to the kind of transformation that they must perform; composeSB and composeTC are two quite distinct functions. The code for this example can be found here:
MonadBind2of3.fsx

Notice that we still have the choice to either combine f and g first and then combine with h, or combine g and h first and then add f in front.

Two at a time

let fghOne = fun x -> composeTC(composeSB(f x, g), h)
let fghTwo = fun x -> composeSB(f x, (fun y -> composeTC(g y, h)))

The Monad is up next

In the next post I would like to eliminate the use of specific transformations done by helper functions. An easy way to do that is to require that the types line up (S = B and T = C), but we don’t want to be quite that strict. Instead we will create a new kind of type, a type that is used as a container for existing types. The new kind of type is a Monad. We will see how three functions that work with types A, B, C and D, can work with new container types R, S, T and U.

Using Monad friendly notation the container types will be defined like this:

Container types R, S, T and U

type R = M<A>
type S = M<B>
type T = M<C>
type U = M<D>

Here “M” designates the Monad that will be introduced in the second part. I expect to post the second part in a week or two from now.

Robert Nielsen, 2014-feb-10.

New free editor for F# (Visual Studio 2013 Express)

Very good news! You may have missed the recent announcement – I almost did. The good news is that now you can use the latest free Microsoft Visual Studio editor to write your F# code (for Windows).

The announcement is found here:
http://blogs.msdn.com/b/fsharpteam/archive/2014/01/22/announcing-visual-f-3-1-1-and-support-for-desktop-express.aspx

I noticed that the FSharp.org page is already updated to reflect this fact:
http://fsharp.org/use/windows/

The trick is, so to speak, to install the new Visual F# Tools 3.1.1:
http://www.microsoft.com/en-us/download/details.aspx?id=41654

And while you are at it why not download “F# Outlining” by Victor Milovanov too:
http://visualstudiogallery.msdn.microsoft.com/bec977b8-c9d9-4926-999e-e50c4498df8a

The significance of the news is that this is the first time since the Visual Studio 2008 (Shell) version(s) that you can get a free Visual Studio IDE for your F# (Windows) coding.

To avoid missing out on this kind of news I can highly recommend subscribing to F# Weekly by Sergey Tihon.

Robert Nielsen, 2014-feb-02.

Type Provider Template from Tao Liu

An easy way to create a new F# Type Provider in Visual Studio 2012 is to use the template provided by Tao Liu.

On Tao Liu’s blog, Apollo 13, there is a post describing how to use the template (it is not difficult – just look at the two images there).

Once the template is installed create a new project, and select “F# Type Provider Template”. I will call the new (project and) solution “HDF5TypeProvider”.

By default four files are found in the solution explorer showing the new project. I want to keep two of these files – “ProvidedTypes-head.fsi” and “ProvidedTypes-head.fs”. The two other files – “TypeProvider1.fs” and “TestScript.fsx” can be deleted as soon as you have finished playing with them.

There are some handy type definitions in the two files that I kept. I am about to add five new files to the new solution. The new files are “HDF5-189.fsi” and “HDF5-189.fs” mentioned in a previous blog post as well as three new files – “HelperFunctions.fs”, “HDF5TypeWrappers.fs” and “HDF5TypeProvider.fs”.

If you right click in the Solution Explorer and select “Add” –> “New Item” you don’t get an option to choose the folder for the new item. Some users think that you don’t get to choose the folder for the new item because you are not supposed to pick a folder yourself. It is quite understandable that you get that impression. In fact it is just some more or less randomly missing feature in the user interface of Visual Studio 2012. Just create the file(s) outside of Visual Studio and click “Add” –> “Existing Item”, and Visual Studio will be happy to add the new file(s) to the solution. There is one caveat more though; by default Visual Studio will copy the selected files to the project folder. I edited the fsproj file in notepad – well actually EditPlus – and it appears that there even is a special icon (a little blue dot or something like that?) to warn that the file locations are outside of the project folder.

Maybe you have already guessed what is in the three new files, but I will give a description anyway, I think, in a future post.

Robert Nielsen, 2013-apr-07.