How to Lose Money Compared to the Null Hypothesis

http://www.starcitygames.com/magic/finance/23507_The_Financial_Value_Of_Dark_Ascension.html (it's premium, "sorry" if you can't read it)

Ben Bleiweiss is obviously a very knowledgeable (if not the most knowledgeable) guy in mtg finance.  However, he is not a particularly good player and his evaluations of cards (and the prices derived from these evaluations) should not necessarily be taken as accurate.  However, in his latest pre-release set review, Ben included a reason you should take his reviews seriously when he devoted a section to evaluating his Innistrad predictions.

"So in short, I was right on 61 cards and wrong on 14. I wish my investment portfolio could hit 50% of the time these days, much less 81% of the time."

This is one of the most misleading statements to be included in any mtg finance article and I hope to debunk it here.

The reality Ben wants you to believe (I didn't include the cards he called bulk since even I have more of a life than that):

Of the cards that weren't predicted as bulk, he got the direction of movement 65% correct (I am too lazy to calculate if this is of any statistical significance, but let's give Ben the benefit of the doubt and say it is.  Also, if you include all the bulk predictions Ben's rate is over 80%).  So if we, as a reader of this fine piece, listen to his advice and choose to buy at pre-release prices if he says the card will go up or stay the same and wait until now to buy it if he says the card will go down, our total cost would be:  $199.75 compared to the $257.25 one would spend if they bought each card at the pre-release price (I am not including the 5 dual lands since there was not individual predictions).  So overall Ben's predictions, on first glance, seem incredible and would have saved the consumer almost 25% of the cost of the set.


What's Really Happening


In reality the only person our hypothetical above would benefit is someone whose goal is to acquire four of each card regardless of when (or if) they actually use that card.  A more useful way of looking at things is to evaluate the purpose of a pre-release pricing guide.
1)  The pre-release is when supply of cards is lowest and demand is the greatest, therefore it is a good assumption that card prices will always go down.  So what happens if we replace all of Ben's predictions with down in the previous example?  We end up with a total set cost of $188.50, a 5% savings compared to  the method utilizing Ben's predictions above.  So by listening advice that you paid for you lose money compared to someone who is under the very obvious impression that prerelease card prices will go down.
2)  Since you don't own any cards in the set yet, knowing the price of a card will go down isn't helpful (since there's no way to take advantage of other people overvaluing the card unless you create some sort of card futures market) so the most important takeaway from a set review is to know the cards that will go up in the future and invest in them.  Here's what would happen if you took Ben's advice about cards that would go up in the future:

For every set of these cards you would have lost $9.75 out of a $24.25 investment (over 40%).  Therefore, by correctly utilizing Ben's predictions, you would have once again paid to lose money.  


It is therefore inexcusable to claim "if you had sold off the cards I said to sell off and bought the cards I said to buy, you would have ended up a lot more ahead than behind"  since this doesn't represent the reality of the situation at all (not even considering the fact that you would not be able to sell any card for scg's sell price).  This is not even including the fact that the card Ben claimed was the "most overrated card in the set" (Geist of St. Traft) is now triple of what Ben's predicted price (and almost double its pre-sale price).  In conclusion, Ben using his Innistrad set review predictions as a reason about why you should trust him about his Dark Ascension review is an outright misuse of statistics and good taste.

-Chris Mascioli
@dieplstks on twitter