Network of Things

I have a sneaky feeling that in coming year, the developer will strike big and get operational control of security in a datacenter and enterprise as a whole. Earlier this year, I warned that this should not  happen. Read This.


But I feel now, it is too late. Here is why. We have moved from securing perimeter to interfaces and now are talking about process jails. Some folks call it micro segmentation moving to nano segmentation.  From a ops person POV this means several orders of magnitude increase in number of endpoints that he has to identify and operationalize. i.e. he cannot do it. It will have to be done by software. And the developer owns software.


Yup, software is eating the world. It just ate the security ops.

Having spent over 6 months now reading and practicing with code snippets on the big data ecosystem, the epiphany came to me that all I was doing was learning a new API. In fact, I was learning three new APIs: RDD, DataFrame and DataSet. It wasn't so obvious when I started reading about Apache spark. See the beauty of APIs is that it speaks the language of a developer and as a developer at heart and training I can easily understand what is being said.

All the stuff I had to read to get to this epiphany about Scale, R, NumPy and in-memory databases, keeping data in CPU registers and not in L1/L2 cache was just all confusion that kept me to from getting to the core. It took six months to weed through so much garbaget to get here.

Ok, so these three APIs help you deal with data. And depending upon the data, you pick one of these. The more structured your data the more you think of datasets and dataframes over simple RDD. And that is all there is to it.