Friday, May 10, 2019

Data is the new Narcotic

About six months ago, I tweeted saying Data is not the new oil, but it is the new narcotic.  A key property of any narcotic is that you need it periodically and with time more of it to achieve onset. Data has the same effect on any cloud based digital platform. Let me explain.

Let's say you are like me - frugual - and shop in big lots. A data at a point in time would be the receipt issued to you that has item's SKU, your loyalty number (for identification) and (most importantly) location/time of the purchase. If the store shared this data with a digital platform, that data would be the first attempt at a drug for that platform. The platform would need this data periodically i.e. your receipt at your next purchase. Why? Because the analytics is done on a time series and insights like time between purchases and second order insights like cough syrup in June means spread of some respiratory infection in that zipcode or if that fails then cross that data with doctor's office visits to arrive at a probability of upper respiratory chronic disease or just smoking. All of these common sense analytics can be done by a computer in seconds, but it needs data to start and it needs to be fed periodically to increase its accuracy. And to arrive at second order insights, it needs more of that data about you. In fact, if all your activities from body functions, to daily habits are digitized and fed to this platform, the smarter it will get on predicting your intentions. To a point that it will predict (serve you an advert) before you have felt the need for it. Like when you wonder how did it know that I need a cough syrup.

They are calling this type of learning where repeated encounters with a data point increases the weight of that happening again (probability) on an underlying learning network:  "AI". And the data is the narcotic that this system needs regularly and in increasing quantities.

Sunday, March 31, 2019

Who killed Blockchain?

We have all been to a notary to complete a transaction or an application for important events in our lives. The general procedure involves one signing/dating the document followed by a signature/stamp from the notary. The notary also makes a entry into his/her ledger. The purpose of the ledger to some time in the future validate the notarization. Now what if we had a technology that automagically places all the millions of entries across hundreds of thousands of notaries into a access controlled ledger on the internet? That would enable notarization by any citizen, entity not in the bad books of society. It would enable cross border notarization (read trade). Extend this use case further and you could apply this technology to validate your credentials, your deeds on your assets, your birth/death/marriage records. This technology would fundamentally change the way humans are organized today.Everybody agrees it would be great to have this technology flourish, but alas I read it obituary everyday.

To understand why, I would invite you to watch the movie who killed the electric car? There is something in the movie for both bulls and bears of blockchain. The bulls will say but wait.. EVs are not dead and so there is hope for blockchain. Bears will say, but those tactics (like filing patents and locking up competitive innovation) worked as it delayed the  EV for several decades. (My own first experience with EV was Sun's Java car in 1999). I think blockchain is going through one of those moments where the ones whose empires will fall ("trusted intermediaries") first joined the party and then played party pooper.

Monday, December 24, 2018

Elephant in the Datacenter

I hear conversations at eateries discussing virtual wires to move packets between software endpoints. I believe the problem of moving  packets or making a remote procedural call  (across administrative domains) is already solved. What is not solved is the policy around the movement and invocations. We need to solve this problem if we are to tackle the elephant in the room.  When I listen to end users of technology, their discuss their pressing problem and that is phishing, robocalls, invasion of privacy and at extreme fear of constant surveillance because of AI. The intrusion they are concerned about is not from rogue states but rogue software that runs within our trusted perimeter. This is not going to be solved at the lower levels of the compute stack. This needs a Layer 8 security solution whose enforcement takes place at all seven (or five) layers below.

I spent the last year trying to understand the appeal of blockchain as a data structure and now I am convinced that is how we should look at blockchain. It is a data structure like a class in Java or struct in C. We should use it when we want to store data with access permissions of the owner of the data. When that happens, we will be uploading our photos and storing our documents with a checkbox which says private and that would cause the platform's software to store the datum in a blockchain that is personal to the person who uploads. It is like a vault in the bank where not even the banker can access the contents without your private key.

This massive misallocation of capital to copy cat ideas only happened because of ridiculously low  cost of money. In a rising rate environment, hopefully we will see capital being allocated on issues to which society is waiting for resolution.

Happy Holidays

Friday, October 19, 2018

(Lack of) Trust is killing the World

Not to take on the "Software is eating the world" meme, I believe the pendulum has swung too far in the SD* world.

Automation was the killer app driving the softwarize everything and give it internal guidance (read ML, AI etc.), but at the end of the day it ended up being a great tool for the bad actors who used the automation to conduct fraud, invade privacy and outright steal. Think of an airport with no security where airlines automate all ticketing, boarding, everything. Not a very secure flight is it? The biggest threat to efficiency/automation/AI is fraud, malice and hmm . bad actors or humans.

What we need now is to build a platform that promotes trust not just communication. I went to engineering school when having an email was a privilege given to certain university students. Today I can communicate with my family over four different chat programs. They all use different ones. Do we really need this? Given all these choices, I still could not locate the person who sold me stuff on eBay that never arrived. What's the point?

If software wants to run stuff, the it should be accountable. Is should be trusted. Currently, the only technology that straddles tech and human boundary is Blockchain. These large platform companies should store all PII on Blockchain which should be under the user's control. My digital recognitions from University diplomas to Employee of the Day award should be on my personal chain. When we get to a point where I get alerted for when someone or something (read SW) is accessing my data using my public key on any platform or within any organization, only then will people trust these newly classified communication services companies. Until then, I think SD* will be put on hold. Enough of efficiency, I just want my privacy back!

Saturday, March 17, 2018

Pets and Cattle

Any pet owner will understand the pain felt by the owner of the pet that died on United Airlines. What essentially happened is that flight attendant dealt with the pets as if it were cattle. Your onboard baggage is cattle and the airline is optimized for cattle. This is not a note on UAL or pets, but on the notion in cloud computing that enterprise applications are pets and should be converted to cattle so they can leverage the cloud computing infrastructure.

The market for enterprise software is around $280B and market for infrastructure that includes servers, network and storage is roughly $60B, $40B and $20B. As cloud takes a large upfront cost hit on the infrastructure they want to host workloads that are more sticky that the ones that drive revenues today. Today's workloads are mostly transient. New workloads start on the cloud and migrate out of it as soon as they are viable (business modelwise). So, it is understandable that cloud computing giants are pushing the pets vs. cattle metaphor and exhorting enterprise IT to migrate to their pet applications to the cloud. And why not, the prize is huge. Almost $200B of revenues should it happen. But will it? The case of UAL and the pet reminds us that pets cannot be treated as cattle and when they are the consequences are disastrous.

As any pet parent knows planning a vacation requires careful selection of pet friendly hotels, airlines and destinations. We don't have the same pet friendliness in a cloud yet. Given the economics of the cloud, it will be difficult.

Wednesday, March 14, 2018

Too many Dashboards

It seems we have a dashboard for every metric and dashboards to pick dashboards. This reminds of the early 2000s when widgets were introduced and we placed a widget for every metric, data stream on the desktop. We have distributed our IT systems and that enables us to monitor everything, but dashboard is not the answer. In fact with so many dashboards I kind of miss the good old monolithic system :)

What we need is an automaton which processes these metrics and takes automatic decisions. Dashboard seems too much like a business process. What would be great is if we get a alert saying "metric reached threshold and controller took some action".. kind of like what we get from our banks when a charge is made or fraud prevented. What would be neat is if we replace a whole bunch of dashboards with a few controllers that execute some policy that was recommended real-time by ML.

Tuesday, February 27, 2018

One horse or 10K Chickens?

Would you like to ride a carriage pulled by 10K chickens or single horse? The answer is not easy. It is technically cool to distribute a job over many compute units and when done successfully it replaces the workload on a more hospitable economic curve (cheaper, faster, better..). However, not all jobs can be distributed and a complicated system which does distribute it over cheap compute units ends up being so complicated that focus shifts from the job to managing this system.

We just spent over 6 mos trying to distribute a webApp and came to the conclusion that it is not about the app but about the data. We need to distribute the data and have compute thread scheduled on the data. But wasn't that the whole point of OOP?

Anyways, the search for distribution introduced me to blockchain. Here is a distributed network that kind of mimics human networks. It keeps people honest and has potential. stay tuned...

Data is the new Narcotic

About six months ago, I tweeted saying Data is not the new oil, but it is the new narcotic.  A key property of any narcotic is that you need...