Estimating confidence intervals, part 2

Here is another data point from my attempt to estimate 90% confidence intervals. This plot shows my daily estimates for completing a feature I was working on.

90% confidence intervals

 

The dashed line is the “truth”: it’s what my estimate would have been if I had estimated perfectly each day. The shaded region represents my 90% confidence estimate: I was 90% confident that the amount of time left fell into that region. The solid line is the traditional pointwise effort estimate: it was my best guess as to how many days I had left before the feature would be complete.

For this feature, I significantly underestimated the effort required to complete it. For the first four days, my estimates were so off that my 90% confidence interval didn’t include the true completion time: it was only correct 60% of the time.

This plot shows the error in my estimates for each day:

 

Error in effort estimate

Apparently, I’m not yet a well-calibrated estimator. Hopefully, that will improve with further estimates.

Presentation as text

I gave a talk last week at Camp DevOps about Ansible and EC2. The talk is written in present format, which is a very lightly marked up text format, similar to Markdown. You can see the source file in a Github repo.

It was liberating to focus entirely on content and not worry too much about the exact appearance of the slide.

I also went for a minimalistic approach where I often didn’t even use titles. The slides won’t make much sense if you just look at them without me talking. Hopefully, they made some sense when I was talking in front of them.

“Who owns the fish” in Alloy

Hacker News linked to a logic puzzle with the following constraints:

There are five houses in five different colors starting from left to right. In each house lives a person of a different nationality. These owners all drink a certain type of beverage, smoke a certain brand of cigarette and keep a certain type of pet. No two owners have the same pet, smoke the same brand or drink the same beverage. The question is: WHO OWNS THE FISH??? Hints:

  1. The Brit lives in the red house
  2. The Swede keeps dogs as pets
  3. The Dane drinks tea
  4. The green house is on the left of the white house
  5. The green house’s owner drinks coffee
  6. The person who smokes Pall Mall rears birds
  7. The owner of the yellow house smokes Dunhill
  8. The man living in the centre house drinks milk
  9. The Norwegian lives in the first house
  10. The person who smokes Marlboro lives next to the one who keeps cats
  11. The person who keeps horses lives next to the person who smokes Dunhill
  12. The person who smokes Winfield drinks beer
  13. The German smokes Rothmans
  14. The Norwegian lives next to the blue house
  15. The person who smokes Marlboro has a neighbor who drinks water

Alloy is very well-suited to solving this type of problem, so I gave it a go. Here’s what my Alloy model looks like:

open util/ordering[House]

sig House {
 color: one Color
}

abstract sig Person {
 occupies: one House,
 drinks: one Beverage,
 smokes: one Cigarette,
 keeps: one Pet
}
one sig Brit, Swede, Dane, Norwegian, German extends Person {}

abstract sig Color {}
one sig White, Yellow, Blue, Red, Green extends Color {}

abstract sig Beverage {}
one sig Tea, Coffee, Milk, Beer, Water extends Beverage {}

abstract sig Pet {}
one sig Birds, Cats, Dogs, Horses, Fish extends Pet {}

abstract sig Cigarette {}
one sig PallMall, Dunhill, Marlboro, Winfield, Rothmans extends Cigarette {}

fact allRelationsAreOneToOne {
 color.~color in iden
 occupies.~occupies in iden
 drinks.~drinks in iden
 smokes.~smokes in iden
 keeps.~keeps in iden
}

pred problemConstraints {

 // The Brit lives in the red house
 Red in Brit.occupies.color

 //The Swede keeps dogs as pets
 Dogs in Swede.keeps

 // The Dane drinks tea
 Tea in Dane.drinks

 // The green house is on the left of the white house
 Green in prev[color.White].color

 // The green house's owner drinks coffee
 Coffee in (occupies.(color.Green)).drinks

 // The person who smokes Pall Mall rears birds
 Birds in (smokes.PallMall).keeps

 // The owner of the yellow house smokes Dunhill
 Dunhill in (occupies.(color.Yellow)).smokes

 // The man living in the centre house drinks milk
 (drinks.Milk).occupies in first[].next.next

 // The Norwegian lives in the first house
 Norwegian in occupies.first[]

 // The person who smokes Marlboro lives next to the one who keeps cats
 (smokes.Marlboro).occupies in (keeps.Cats).occupies.(next + prev)

 // The person who keeps horses lives next to the person who smokes Dunhill
 (keeps.Horses).occupies in (smokes.Dunhill).occupies.(next + prev)

 // The person who smokes Winfield drinks beer
 Beer in (smokes.Winfield).drinks

 // The German smokes Rothmans
 German in smokes.Rothmans

 // The Norwegian lives next to the blue house
 Blue in Norwegian.occupies.(next+prev).color

 // The person who smokes Marlboro has a neigbor who drinks water
 (drinks.Water).occupies in (smokes.Marlboro).occupies.(next+prev)

}

run problemConstraints for exactly 5 House

Alloy’s “Magic Layout” did a surprisingly good job at displaying the results. I had to manually rearrange the output so the houses would be displayed in the correct order, but otherwise no fiddling was required. Here’s what it looks like:

alloy-fish

I also put it up on Github.