Dependency reconstruction using Infer.NET

Today we'll take a look at the sexual relationship problem from another point. Let's assume that there is some decision model describing how girls select their sexual partners. But, unfortunately, we don't know that model yet. One day some girl we know has betrayed a secret: decision model is linear and it depends on such a factors: your sexuality, your smartness and also girl's slutness. So we decided to find out the way that model actually looks like. The most simple way to do that is to gather some training data of the following format: (sexuality, smartness, slutness, was there any sex). Yeah, we assume that we can somehow estimate that values for some girls and boys and ask them about sex. Than we can use Bayes point machine implemented in Infer.NET to reconstruct that linear model. Factor graph for that implementation is really simple. It just assumes that we can get an answer by calculating inner product between weight vector (which describes our linear model) and observation vector (which contains first 3 values of the training data vector followed by 1 for representing bias) and checking if it is positive:

That model can be build using following code piece:

Range range = new Range(ItemsCount);
VariableArray<bool> observedSex = Variable.Observed(observedResults, range).Named("observed sex");
VariableArray<Vector> trainingData = Variable.Observed(observedData, range).Named("observations");
Variable<Vector> weights = Variable.Random(
new VectorGaussian(new Vector(ParamCount),
PositiveDefiniteMatrix.Identity(ParamCount))).Named("weights");
observedSex[range] = Variable.InnerProduct(weights, trainingData[range]) > 0;

So, let's generate some observed data using "unknown" linear model:

const int ParamCount = 4;
Vector[] observedData = new Vector[ItemsCount];
bool[] observedResults = new bool[ItemsCount];
for (int i = 0; i < ItemsCount; ++i)
{
double sexuality = Math.Max(0, Math.Min(Gaussian.Sample(0.5, 1), 1));
double smartness = Math.Max(0, Math.Min(Gaussian.Sample(0.1, 0.5), 1));
double slutness = Math.Max(0, Math.Min(Gaussian.Sample(0.3, 1), 1));
bool haveSex = sexuality + 0.1 * smartness - slutness > 0.25;
observedData[i] = new Vector(sexuality, smartness, slutness, 1);
observedResults[i] = haveSex;
}

We can now infer weight vector posterior distribution using Expectation Propagation algorithm. Gaussian distribution is symmetric and unimodal, so we'll take it's mean as a result.

InferenceEngine engine = new InferenceEngine { Algorithm = new ExpectationPropagation()};
VectorGaussian weightDistribution = engine.Infer<VectorGaussian>(weights);
Vector mean = weightDistribution.GetMean();
Console.WriteLine(
"Model: ({0:0.00})*sexuality + ({1:0.00})*smartness + ({2:0.00})*slutness + ({3:0.00}) > 0",
1, mean[1] / mean[0], mean[2] / mean[0], mean[3] / mean[0]);

After running that sample we'll get the following output which confirms that we have reconstructed unknown linear model quite precise:

Compiling model...done.
Initialising...done.
Iterating:
.........|.........|.........|.........|.........| 50
Model: (1.00)*sexuality + (0.10)*smartness + (-0.99)*slutness + (-0.26) > 0