Bigdata and Melman’s Hypothesis

Let us revisit our friends from Madagascar again. We find them stranded in the island nation of Madagascar, not knowing where they were.  Their conversation goes like this –

Yeah, here we are.
Where exactly is here?
San Diego.
San Diego?
White sandy beaches, cleverly
simulated natural environment,
wide open enclosures, I'm telling
you this could be the San Diego zoo.Complete with fake rocks.
Wow! That looks real.

Last time I wrote this about Melman’s assertion

A less forgiving view of Melman’s behavior will be that he started with a preconceived notion and then looked for evidence that supported his notion, ignoring those that would contradict it. Once we have made up your mind our cognitive biases nudge us to only talk to those who would support us and ask only questions that will add credence to our premise. No wonder he did not consider the fact that they were on sea shore and the San Diego zoo doesn’t open up to the sea (and the equatorial climate etc).

Let us see how their discussion will go in Bigdata world,

Yeah, here we are.
Where exactly is here?
San Diego.
San Diego?
White sandy beaches, with billions of grains of sand
look at the sand here
look at the sand there
look at sand over there
...
...
all these billions of grains of sand say this is San Diego

Unfortunately adding tons of sand (or petabytes of data that carries no additional information and has significant self-correlation ) does not help if you started with flawed pre-conceived notion and fail to ask what data will falsify your original claim.

Even with BigData you are stuck in Madagascar thinking it is San Diego.

Don’t just add data that make Madagascar look like San Diego. Seek data that will show why it is not.

Data, Data everywhere. And not an iota to help with decision making.