Raahul's Blog

In the early 2000s, a lot of learning happening on IRC. You’d befriend a few people, and we’d all teach each other something that we know or have learnt. From writing small games in GameMaker (now part of YoYo Games) to writing extensions in C++ and assembly, there were kids who were practically doing everything. Actually, that’s where I first heard about neural networks, though I didn’t study it in detail.

Back then for me, programming was about creating something. Today, it’s spent inside the IDE wrestling problems that are not even related to the domain, like endless code organization, naming stuff, etc. These kind of secondary problems are fun when we’re still learning aspects of programming/software engineering, but later, they become an annoyance.

This dealing with code that’s not directly related to your domain is what I call code oriented programming. If the data that you’re dealing with is the hero of your code, then I call that data oriented programming.

In this Clojure Made Simple talk, Rich Hickey mentions this concept:

The majority of functions in Clojure just take data, and by data I mean immutable, unadorned stuff, and they return that same thing. A giant library of functions. Hundreds and hundreds and hundreds of data manipulation functions, ready to go. They take data. They return data.

So if you have anything that is data, you can use all those functions on it, which tends to press you towards making everything data.

-Rich Hickey in Clojure Made Simple (source)

Today, especially in frameworks like Ruby on Rails, the database is so coupled to the application that the application drives the data structures, not the other way around. Things like “polymorphic relationships” are easy and fun to have until you have to break the monolith and a service uses something other than Ruby. I remember the endless integrity problems that I ran into because developers did not know that a relationship in Rails does not mean a FK constraint.

Even as early as 2012, my friend and I would spend time designing the database more than we spent designing the application. But now, in just a few years, the situation has reversed.

I would not say it’s worse today. Code oriented development can be very good for a lot of tools, but as soon as you become an enterprise that is going to be around for ages, then thinking data oriented becomes a very valuable option. Your code would evolve, but the information conveyed by your data evolves much slower than that and unfortunately has more rigidity.

Meetings are the new-age power naps of the 21st century. Everyone is in it, but no one seems to get any value out of it.

Have you ever been a silent spectator in too many meetings? Did you understand neither the problem nor the agreed upon solution? Was there even a problem? Did everyone even come to an agreement?

The first problem: no agenda

“Hey, I’m calling this meeting to discuss feature XYZ.” Unless you’re on a panel on TV, that statement makes very little sense. Discuss what? Why? What’s the core problem that needs solving?

A lot of meetings have no agenda. Unless you’ve clearly marked the summit of the mountain that you intend to climb, no one has any idea which direction they should all be going.

Have a clear agenda. “We want to start development on XYZ, but we need to finalize the API contract.” This is much better, although doesn’t probably require a synchronous meeting. However, at least the meeting has an outcome of unblocking the development of a feature.

The second problem: too many synchronous meetings and no context

“We’re here to finalize the database schema for our Dropbox clone.” Okay, but is that going to happen in a 30m meeting? Did you even send out a documentation before so that people could have some context before coming into the meeting?

Also, if you have people who are tasked with database design in the company, just assign a new story to them to verify and adjust the schema. And also give them all the context. They’ll be able to use their own time & process to come up with the design, without the distraction of a meeting.

In short, managers should delegate tasks to people who are good at it, and just get out of their way.

Having started programming in the early 2000s, I was used to having sparse resources for learning anything. Fast-forward 10 years, as I was in my 3rd year of engineering, I came across an excellent programming course by Stanford’s Mehran Sahami called Programming Methodology. Although I was already proficient in programming by then, but I still watched the videos and did the assignments because the instructor was so captivating.

This course takes someone who has never done any programming, and then turns them into productive programmers. By the 3rd programming assignment (Lecture 10), you’re creating a breakout game. That’s amazing progress of a class full of newcomers.

Years later, when I actually got a tour of Stanford from a friend who graduated from there, I asked him, “Have you heard of Mehran Sahami?” (well, he was the only Stanford professor that I knew about). He replied, “His classes are one of the most popular.” Which was to be expected given his amazing style and the way he could communicate programming concepts.

If you’re a newcomer who has learnt programming only through online blog posts and stuff, this course could help you unlock your ability. You probably are capable of creating much more complex programs than what these online tutorials make apparent.

Finally, don’t be put off by Java. This course is valid even if Java is not going to be your primary language in the future.

I’ve noticed that a lot of times, the beliefs and conclusions of people (in tech, because that’s what I’m exposed to the most) are the sum of first page results from Google.

When out of curiosity I debugged an issue with rendering performance in a ReactJS app, I observed that although we used ImmutableJS, we were just initializing a new immutable object each time we got a new API response. We were not using the concept of persistent data structures to make our rendering more efficient, which is why it was taking about 5s for the whole app the render after getting the API response.

Well, someone decided that ImmutableJS was the way to go because it was trending at that time. The homepage looked decent too. As long as we’re using it, it doesn’t matter for what.

NoSQL’s “marketing” around performance was very effective for a certain demographic. SQL didn’t scale for you? You probably didn’t screw up the design, you just need a NoSQL solution for it. I know a team that moved from MSSQL to Mongo because NoSQL would potentially offer better performance. For a database few tens of MBs and not growing that fast.

Even during my initial days as a programmer, I held a lot of beliefs that were really not mine. Even I was guilty of it in retrospect.

Since then, I try to experiment around and verify things for myself, although it is very cumbersome. So it’s really important to prioritize.

As an individual, you probably have a lot of ideas that are in variance with the collective popular belief. And most likely, you might be on to something, because you’re at the very least thinking for yourself.

Proud dropouts sometimes swear by online courses, because the project-based learning that they provide is hands-on, and also representative of the how the real world is going to be. On the other hand, proud academics sometimes like to brag about equations and theorems that are really not relevant, but still sound academically enlightening.

I’ve done numerous Udacity nanodegrees and paid Coursera courses. I’m also currently pursuing my masters online from Georgia Tech in computer science (it’s OMSCS program). I’m going to make a contrast between the two approaches, highlighting their merits and demerits.

Vocational education can be very ineffective over time

I remember sitting with a distinguished security expert in my company, along with a colleague of mine. The security expert asked my colleague, “Which programming languages do you know?” He answered, “Ruby, mainly focusing on Ruby on Rails.”

At this point, the security expert said, “It’s the way you answer the question that tells a lot about your experience.”

Although he did not elaborate, I kind of knew what he was talking about. That’s because I’ve been asked that question repeatedly in the past, and each time my response would be, “Well, it’s a tricky question to answer. I can learn the syntax of any new language pretty easily, but the eco-system is an entirely different thing. What exactly do you mean by knowing a language?”

Only vocational learning results in people learning a vast amount of individual facts, but have no idea of how they tie in together. The abstraction that ties facts together is theory, but having no exposure to theory can sometimes impede your progress.

You might keep learning a new JavaScript framework each month, but in the grand scheme of things, you’re not really learning anything new. It just seems that way. What I learnt in Udacity had a very short shelf life. Tensorflow? Well, it’s PyTorch now.

College education is useless if you can’t use it

The one thing that most of the colleges don’t do well is teaching you how to apply what you just learnt. Also, the claim that colleges teach you how to think about a subject is also grossly false. The only advantage is that you get exposed to theoretical backings (which is basically other people’s way of thinking about something) for different subjects. If learnt correctly, the concepts don’t change much over time, although programming languages might.

Want to learn how to read technical papers? Well, they might have an assignment that’d ask you to do that. But the thing is, the college doesn’t teach you how to do that. It just tells you to do it. And you end up forcing yourself to figure out how.

“Here’s Bayes’ theorem. And here’s 100 more theorems on top of it. You don’t have to memorize them. We’ll teach you how to derive them. But those derivations you might have to memorize.”

I’m more interested in how the first person thought about these ideas, not a sugar-coated interpretation of it ages later. Although college education taught me the “what” of these mathematical rules, I’ve never seen an instance of “why” and “why only this way?”

In conclusion

Both in Udacity’s case and Georgia Tech’s, the assignments were the only enlightening part. The theoretical videos were just about “relaying information” rather than increasing knowledge. Unless you’re already familiar with a subject, you won’t be able to understand it to your satisfaction through both these mediums.

The best way to learn is to do your own experiments. Once understood, that understanding lasts a lifetime. Facts can change, but the governing rules, if deciphered, won’t.

Also look at: The best way to learn Python is not always courses

When preparing for my first violin group performance, my teacher gave me a valuable advice that is applicable everywhere. He said, “Remember, you’ll only play 25% of what you’ve practiced on stage. There’s going to be distractions, the crowd making noise, your fellow players making mistakes, and also the fact that you can’t stop and correct your own mistakes. You just need to keep going.”

The core idea is, don’t try to do cutting edge stuff on stage if your objective is to give the audience a great performance. Practice at home, and play on stage only things with which you’re absolutely comfortable.

The usual over-enthusiastic software disaster

Imagine finally getting HBase into production. Your 45MB of big data finally has a new, comfortable, scalable, and cool home. But what happens to houses built on shaky foundations?

They eventually crumble. And that’s because you don’t probably know all the gotchas with it. Disaster recovery can get tricky even for known technologies, but with the known, no one is going to take responsibility for your failed experiments.

Just like medicines and surgical procedures that you go through have been verified to some degree and standardized, your skills need to be the same. You are generally not given experimental drugs unless you’re part of a trial.

Similarly, don’t experiment when the product requires straight results. If you endlessly experiment with a CRUD application, then you’re not effectively applying what you’ve learnt so far. Your experience becomes useless, and this creates a lot of grief down the road.

Remember, when you experiment, you often don’t have a Plan B. And Plan B, C, Ds… are very important to have because things do go wrong, and your customers won’t always be sympathetic to these disasters.

Never say never

This doesn’t mean you should never experiment. It only means that everyone should be on the same page about what’s an experiment and what’s a sure-shot expectation. Software can get complex, mostly because of the collaboration and long-term maintenance required. But some things should be simple to do after so many years, and we should keep them simple.

Shortly after graduating college in 2012, I decided to learn violin, which is a very demanding instrument. My hobbyist knowledge of guitar and the piano did help, but not much. I used to go to a class for 2 hours every week, which seems very little. It seemed as if it’ll take forever to learn anything decent to play.

Fast-forward about 2 years, by which time I had won proficiency awards and had done a group public performance during festival season. All this while, I was also learning the flute, in which I also got several proficiency awards and got to do a group public performance.

Was our training intense? Well, violin does require intense practice. However, our lessons were to the point. Once you have a great teacher, he will guide you to practice things that really are important, and ignore a million other things that are doable but not worth doing. Prioritization was the name of the game.

The intensity was not because we did a lot of things under serious time constraints, which is what we irrationally keep doing nowadays when we want to learn anything new. But it was the deliberate, consistent, concentrated, and reasonably manageable efforts that we put every day towards an exercise.

This syndrome of trying to cram in a lot of material is especially visible in this “machine learning & data science” age. You see tutorials and courses that will teach you everything under the sun about ML, but you will actually not understand any of it, nor will you able to adapt these learnings to your situation.

I recently read The Compound Effect by Darren Hardy and it gives you a lot of examples into this core idea of doing small things consistently, rather than going after a windfall.

After programming for ages, attending every known coding conference on the planet, and wrangling your way through software engineering practices and building large scale systems, what becomes your biggest obstacle to effectiveness? Idiotic companies and their hiring practices.

We need a junior Golang developer for an amazing large-scale platform (translation: CRUD on more than 10 records). You should be able to write expert level go, even though you’re being hired at a junior position, which is just our way of saying we’re not going to pay you much. If you’ve created your own world-famous programming language, then that’s a plus, and we might just give you your own computer to use.

Expert Ruby on Rails programmers needed! Join us as unpaid interns. You should know how to create large scale applications like the Google search engine (seriously? using RoR?). We need people who understand the fundamentals of algorithms well so that they don’t make any mistakes while running the Rails scaffold or while adding a new route to our already CRUDed (read crowded) app.

Don’t have unreasonable expectations

If your company doesn’t use this crazy new algorithm that you just studied about last night, don’t ask about it. Assess candidates’ fitness to the role by simulating real-world situations, especially things that happen every day. This is a good way of being transparent.

If you find someone who is able to answer your crazy question, then he is smart enough to understand that your company is really not using any of it.

Be respectful to people you’re interviewing

I’ve taken countless interviews, and the lessons that I’ve learnt are:

1. Always be nice, kind, and dignified. Be petty in your own time.
2. Understand that people come from different companies that have different priorities. What’s a best practice in yours might not be a best practice in theirs.
3. Ask questions out of curiosity and with an intent to learn, not to show people how much they don’t know.

There are good tests like fizz-buzz that can be administered early on, so that you don’t waste anyone’s time. But it’s important to not take it to the extreme and start asking questions that no one even in your own company can answer.

Look at this quote by Russ Cox:

Software engineering is what happens to programming when you add time and other programmers.

Doing incremental changes to a program, without any consequence of future maintenance (time) or other programmers (collaboration) is laughably easy.

Want to add a new database column? Why not? Which database should be use for our 10MB of data? I heard Cassandra is very scalable!

Well, whatever you do, it will most likely work. It could be over-engineered, cause maintenance hell, make your application slower over time, but as long as you’re only doing local optimization (trying to get just this one thing in), anything would most likely “work.”

The key skill that will make you an established engineer

Having talked to a couple of CTOs and senior engineering leadership over the years, the one skill that is often found lacking, though highly desirable, is communication. People might know how to talk, but that’s not communication.

Communication is the act of sending something over a distance, and making sure that what was sent is exactly what arrives

A lot of people interpret communication to be just sending something over, but they don’t take responsibility for making sure what arrives is exactly what was sent. Without communication skills, you can neither have empathy for others, nor allow others the opportunity to have it for you.

How communication skills make you a better engineer

Communication skills are not synonymous with the English language or even having an exceptional vocabulary that no one understands. It extends to even the act of programming.

1. Are you able to read the intent of a fellow programmer by looking at her code? Reading code can be a surprisingly difficult endeavour.
2. Are you able to understand the DevOps’ pain in managing different systems, and thus you choose to use MySQL everywhere? Sometimes, it’s better to be uniform than the usual unqualified “use the best tool for the job” nonsense. Here, best often refers to someone’s preference.
3. Do you end up using PHP or Java because those are the only two skillsets easily hireable in your area?

And there could be a lot more factors. None of them are covered in your regular programming courses that teach you how to write the for loop 10 different ways in 20 different languages.

Remember, programming is a tool. Engineering is a purposeful use of science. And that science is not always just computer science, but any other science that backs your domain too.

Here’s a concise resource to learn the basics of Python quickly: https://docs.python.org/3/tutorial/. It’s official, concise, and covers some major aspects. It also spends less than 5 minutes on if and for statements, as opposed to devoting hours to them. (If you read this post till the end, I’ll show you another invaluable and lesser known resource for learning specific language features of Python in detail.)

My suggestion for going through the Python Tutorial above is to read the first 5 chapters, from “Whetting Your Appetite” to “Data Structures” in one go. You may not may not understand a few things, but that’s okay. At least make note of things that you do. Then, be more meticulous in the 2nd pass, and then some more in the 3rd, etc.

Knowledge is like a network, you have to make multiple passes over the material to start making your own connections, similar to how algorithms like PageRank converge. (Now that might be a good motivation to go look up PageRank if you’ve never done so in detail)

Free, detailed, before-date guides for Python language features

Consider “assignment expressions,” a feature slated for 3.8. How do you learn it? Why was it introduced? How will it benefit you or other programmers? How will it ultimately save your time?

You have two options to answer those questions. Either wait for your favourite blogger or YouTuber, or just go and read the PEP: https://www.python.org/dev/peps/pep-0572/. A PEP is a Python Enhancement Proposal. You can see an index of it here: https://www.python.org/dev/peps/.

This is where the proposer argues why this feature is needed, and how it will improve or ease the programmer’s efforts. This is the most direct source of answers for all the questions above.

In conclusion

Use the Python Tutorial to learn the basics, and use the PEP to get a deeper understanding and keep yourself updated. Software engineering is not always about just code, and PEP might give you the additional useful context.