What He Said – Tim Bray · Software in 2014

Tim Bray has a great post discussing the state of software development in 2014.  I found myself nodding in agreement virtually all the way through this one, mainly about client-side development for mobile and the web.

The client-side mess · Things are bad. You have to build everything three times: Web, iOS, Android. We’re talent-starved, this is egregious waste, and it’s really hurting us.

A bit about mobile:

Mobile sucks · I’m going to skip the differences between Android and iOS here because they’re just not that significant in engineering terms. Anyhow, here’s the suckage.

  • First of all, you have to do your mobile development twice.
  • The update cycles are slow. Days in the case of iOS, hours for Android (compared to seconds for browser-based apps). What’s worse is that you can’t even count on people accepting the mobile-app updates you send them. Got a critical data-losing account-compromising privacy-infringing bug? Sucks to be you.
  • The devices are memory-starved, CPU-starved, and battery-starved.
  • There are loads of form factors and it’s getting worse.
  • You don’t get a choice of languages; if you hate both Java and ObjC, get another job.

Bottom line, client-side development is a difficult place to live but server-side is more stable.  I have felt this way for a long time, client-side makes me cuss and server-side makes me smile.  

It’s a good read.

Maybe You Are Doing Too Much Test Driven Development

I came across this StackOverflow post from 2008 where someone wanted to know when a developer can do too much testing.  Simply brilliant summary by the father of test-driven development (TDD), Kent Beck:

I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence (I suspect this level of confidence is high compared to industry standards, but that could just be hubris). If I don’t typically make a kind of mistake (like setting the wrong variables in a constructor), I don’t test for it. I do tend to make sense of test errors, so I’m extra careful when I have logic with complicated conditionals. When coding on a team, I modify my strategy to carefully test code that we, collectively, tend to get wrong.

Different people will have different testing strategies based on this philosophy, but that seems reasonable to me given the immature state of understanding of how tests can best fit into the inner loop of coding. Ten or twenty years from now we’ll likely have a more universal theory of which tests to write, which tests not to write, and how to tell the difference. In the meantime, experimentation seems in order.

I always find interesting to read so many blog posts about how to do TDD, including unit tests, integration tests, functional tests, factories, mocking and on and on.  There are conferences dedicated to software quality and software craftsmanship where TDD is advocated as all-in and anything less is not doing the right thing.

Clients pay me to write code too and not tests.  I write tests just enough to know the critical pieces of my system are working….and continue to work.  Developers could easily spend double their time writing completely tested systems and I’m not sure the customer gets 2X the value or ROI.

I think experimentation is certainly in order and see what best fits your situation.  The biggest takeaway, step back and decide for yourself what’s the right amount of testing and don’t simply follow the pied pipers of TDD.   Lots of great feedback from the others in response to the original question in that thread and great food for thought.

UPDATE: If you want to hear another developer’s perspective on TDD and using it as-needed, Marco Arment discusses how he uses testing in his very successful business on the latest Build and Analyze.  Very timely to my post.

Functional Programming Principles in Scala

Large icon

I subscribe to the school of thought that learning should never end, and I also believe that as a software developer it’s important to learn new programming languages yearly.  

After learning about Coursera and hearing that Martin Odersky was going to be teach a course covering functional programming titled Functional Programming Principles in Scala, I had to sign up.  The class is a 7 week long introduction to functional programming principals using the Scala programming language.

Course Syllabus

Week One: Programming paradigms; overview of functional programming and the Scala programming language.
Week Two: Defining and using functions, recursion and non-termination, working with functions as values, reasoning by reduction.
Week Three: Defining and using immutable objects, review of inheritance and dynamic binding.
Week Four: Working with collections: Sequences, sets and maps
Week Five: Defining recursive data and decomposition with pattern matching.
Week Six: Reasoning about functions
Week Seven: Case study

Weekly

Each week, on Tuesdays, students are presented with a series of lectures by Martin that cover the week’s topic.  The lectures include prepared slides, writing on a virtual whiteboard and live code examples and run 2-3 hours per week.

After listening to the lectures for the week there are homework assignments which exercise the material reviewed in the lectures.  Pretty common for a college course.

I have to admit the assignments are hard.  I am always up for a challenge with a new language but the assignments go well beyond general Scala syntax or basic functional programming paradigms.  These lectures force the student to learn (or remember) computer science and mathematics from my early days of algorithm and data structures from college.  No complaints here, all really good stuff, but it took some research to recall some of these things.

Conclusion

Would I recommend this class?  You bet!  It was a great class and I learned ton. Scala was a language that I wanted to have a look at and this course was a great introduction but assumed a background in programming and did not cover the basics.  

One things that annoys me is that most programming books start with the very basics of a language, trying to cater to the widest audience possible.  This leaves most books with only 1/2 of their content really unique and usable.

After spending some time with Scala, I decided I was not a fan and won’t be using the language for future projects.  I love the idea of functional programming, lots of small functions with specific functions, but Scala is way too verbose for my liking.  It just looks like functional Java and not different enough from Java or C# for that matter to attract me to use it.  I think Scala drove me to appreciate Clojure much more and I will be writing a lot more parentheses because of it.  

I think it’s good to see other programming languages and styles.  Scala works for some but not all.

Functional Programming Battle

2012 10 03 09 09 21

I’ve been spending some quality time with Scala lately and ran into this research paper comparing Scala, Haskell and F# titled Haskell vs. F# vs. Scala: A High-level Language Features and Parallelism Support Comparison.  It’s a great read and well-worth the time.

This paper provides a performance and programmability comparison of high-level parallel programming support in Haskell, F# and Scala. Developing several parallel versions, we employ skeleton-based, semi-explicit and explicit approaches to parallelism. We focus on advanced language features for separating computational and coordination aspects of the code and tuning performance. We also assess the impact of functional purity and multi-paradigm design of the languages on program development and performance.

It’s interesting to see the three languages compared for their abilities in parallel applications.  It should be noted that the paper is from the Glasgow Parallel Haskell web site so there may be some bias but nothing outstanding.

I like the fact that the comparison is based on a Java VM language, Scala, a .NET language, F#, and a native language of Haskell.  Running on a rich runtime such as the JVM or .NET gives the developer a library of which to draw from, it also adds some overhead.  The results are pretty much what I expected with Scala doing better than I would have envisioned.  The JVM seems to be well-tuned. 

You can download a PDF version of the paper here.   The code used in the benchmarks are also available.

Read Structure and Interpretation of Computer Programs for Free on the iPad

9780262011532 medium

I found out recently that one of the all-time classic book in computer science, Structure and Interpretation of Computer Programs, is available in ePub format for free.  The book is truly a classic but has been out of print for a long time now.

For those unfamiliar with the text, from Amazon’s site:

Abelson and Sussman’s classic Structure and Interpretation of Computer Programs teaches readers how to program by employing the tools of abstraction and modularity. The authors’ central philosophy is that programming is the task of breaking large problems into small ones. The book spends a great deal of time considering both this decomposition and the process of knitting the smaller pieces back together.

The authors employ this philosophy in their writing technique. The text asks the broad question “What is programming?” Having come to the conclusion that programming consists of procedures and data, the authors set off to explore the related questions of “What is data?” and “What is a procedure?”

The authors build up the simple notion of a procedure to dizzying complexity. The discussion culminates in the description of the code behind the programming language Scheme. The authors finish with examples of how to implement some of the book’s concepts on a register machine. Through this journey, the reader not only learns how to program, but also how to think about programming.

The contents of the book are timeless and give computer scientists young and old a solid level of knowledge.  It uses Scheme, a dialect of Lisp, but that should not deter readers as the lessons are universal.

The full text is also available to be read online for those not able to read in ePub format.  Great stuff!

Dark Matter Developers: The Unseen 99%

A really interesting and true post today by Scott Hanselman as he explains what Dark Matter Developers are and how a large percentage of developers can be classified this way:

My coworker Damian Edwards and I hypothesize that there is another kind of developer than the ones we meet all the time. We call them Dark Matter Developers. They don’t read a lot of blogs, they never write blogs, they don’t go to user groups, they don’t tweet or facebook, and you you don’t often see them at large conferences. Where are these dark matter developers online?

Scott points out where he thinks these developers are:

Where are the dark matter developers? Probably getting work done. Maybe using ASP.NET 1.1 at a local municipality or small office. Maybe working at a bottling plant in Mexico in VB6. Perhaps they are writing PHP calendar applications at a large chip manufacturer.*

Personally, I know quite a few of these developers and I prod them to not be so introverted, get out to meet ups, blog or at least join and use Twitter.  Maybe these folks have it right, maybe we should just get things done and spend the rest of the time focused on things that are more important to us.  I think they could be right, we spend our time on things we are passionate about, we work because we have to…we need to provide for our families.

To me, the difference between the Dark Matter Developers and those like Scott, is the passion.  Not everyone is passionate about the work they do, the work on and move on to things they like better.  I have been passionate about computers for as long as I can remember, devouring technology in every waking hour because it fascinates me.  I believe this is what separates these two distinct types of developers.

I am a Dark Matter Home Owner.  I hate doing maintenance around the house; lawn mowing, raking, building and painting.  I have better things to do with my time and fixing a leaky gutter is not it.  I would rather read a book on algorithms and pay someone to mow my lawn..that’s just I roll.

Enough TextMate Talk, Go Write Some Code

It seems every few months I hear another diatribe about using Vim or Emacs and switching from TextMate.  The usual reason for switching from TextMate is because version 2.0 hasn’t shown up yet.  So what?

Watt Martin has been providing some interesting and unbiased thoughts on text editors lately with a couple of interesting posts.  The first, Text Editor Intervention:

I love you all, but it’s time to stage a text editor intervention. Put down the mouse, back away from that download link, and take a deep breath.

First off: if you are a Mac user and compatibility with TextMate is an absolute must-have, let me ask you two questions. Is TextMate 1.5 still working for you? Can you keep living with its limitations? If you answered both those questions “yes,” our work is done here. Go in peace.

I use TextMate daily, it does everything I need.  I don’t need to code faster or have a half-dozen split screens open.  My aging mind can only process code so fast and having split screens open, is not going to help me.  But that is just my preference.  Many new editors have popped up on my radar recently like Sublime Text and Redcar, which refer to their TextMate support as Watt points out:

Those of us who cling tenaciously to TextMate do so because of its amazing bundle system; you really have to spend time digging into it to understand how powerful it is. BBEdit, Vim and Emacs all require you to learn a weird scripting language; TextMate lets you write in any language you can script a Unix shell with, from bash to Ruby.¹ The flip side of that, though, is that TextMate’s bundles are—with few exceptions—limited to passing standard input and environment variables to a shell script and getting standard output back. With BBEdit and the Ugly Unix Twins, you can attach scripts nearly everywhere, with deep access to the underlying editing engines.

The new editors popping up are attempting to be the next TextMate, to attract the users now running TextMate.  It is a testament to TextMate, trying to imitate what works.

If your editor works fine for you and gets the job done, why do we feel pressured to move on to something else because it is under more active development?  Actually, I regularly see new minor updates for TextMate.  People were promised version 2.0 and because it hasn’t arrived they feel compelled to jump ship.  This is probably the single silliest reason I have ever heard of for subjecting oneself to a steep learning curve to a new editor.

My opinion about text editors is ignore everyone and go with what works for you.  I’ve tried MacVim, but wasn’t productive enough up-front for my taste and there was no pressing reason to switch.  If I had to, I would probably commit to the effort of learning Vim once and for all.

Stop worrying about when TextMate 2.0 will be out and go write some code already.

Writing with Agile Style

143571757 ba4415ea00

I love writing.

When I am not writing software I like to step away and work on something for this blog or for something more involved for InfoQ.  I often think I might like to do it full-time if I could make a reasonable living from it, but that’s unlikely.

I didn’t realize what felt so good about writing until recently when I was thinking of post topics.  I get a thrill from pushing a new post or article out, a real sense of satisfaction.  When hitting the “Publish” button I get a true sense of accomplishment, that I have added just a bit of value to the world.  This is what makes Twitter so powerful, 140 character maximum posts at a time.   Quick and easy.

I think the satisfaction equates to coding as well.  I will be the first to admit I sometimes wait too long to get a set of features out, just holding off for “one more thing”.  As we have all heard before, release early and release often.  I know when I have a quick fix and get it out, it feels good, so why not do this all the time.  This is what the Agile folks preach.

So, it’s important to get stuff out there, off your plate.  The blog post doesn’t have to be nominated for a Pulitzer Prize, that code doesn’t have to be an entirely new application.  Small wins can be big for your confidence and motivation, they are for me.

Where Are We Without Community?

communityI came across a really interesting blog post yesterday with a recurring theme.  It was by developer Dave Newman titled, Leaving .net.  Does it sound familiar?  It does to me and after reading Dave’s post, the reasons are a familiar sound.

So many .NET shops have their heads buried in the sand, developing mounds of software with antiquated principles:

A company I worked at once had been operating for years under the assumption that it was OK for software to take months to integrate and deploy; they had never heard of continuous integration. They thought it was normal to rewrite your application from scratch every 2 years; they had never heard of Test Driven Development or iterative development. These techniques have been proven to significantly raise the bar on quality, cost of development, cost of maintenance and general morale. Turns out you can still find, in this day and age, software practitioners who haven’t even heard the terms.

It has been shown that Test Driven Development (TDD) improves code but it seems the .NET community fails fully embrace the concept.  I have been part of many .NET projects where stakeholders simply refused to spend the time doing TDD, not when deadlines need to be met.  They were simply too naive to know that deadlines would still have been met but with better code.

My last .NET project was led by a developer who was the master of “not invented here” syndrome.  The project was a unique eCommerce application originally architected 5 years earlier and consisted of some complicated code.  The developer, who didn’t want to spend the time to learn the system, wanted to rewrite the code his-way so he could understand it.  His way was easily worse and did not take into account all the side effects his way may have introduced.  I am glad I am not the only one to witness this:

I have also seen on numerous occasions developers build their own libraries and frameworks to solve well understood problems in curiously terrible ways. The endless rebuilding of data mappers, logging code, object resolution mechanisms, messaging systems and web frameworks adds up to a massive waste of time, effort and potential. I’m not talking about public collaborative efforts, I’m talking about in house “Not invented here” syndrome. This is what happens when you have a community cut off from the world. A community that doesn’t talk to each other and work together. This is what the .net community is.

Communities exist everywhere but none stronger than those in the open source world.  I can’t think of a more diverse and deep group of developers than those using Ruby.   Developers in this community rally around each other, support each other and the projects they create.  If there is no support, no community development the project simply dies off.  One can use Ruby on Rails as a great example, go ready it 1600 contributors.  This is project done for the community by the community, not reinventing the while but improving it.

Has Microsoft ever done anything like this?  No. The closest I can think of is IronRuby and they have since turned their back on that project.   Microsoft tends to treat developers as something to be fed, who pushes out updates as fast as possible to get the pack on to the next thing.  If you look at Silverlight going from v1.0 to 4.0 in a blaze or WCF moving fast to its current state, developers can keep up.  Most of the time we can’t keep up because we are fed and are not allowed to also cook, like in the Ruby community.

This is the same with ASP.NET MVC:

Last year Microsoft released a new web framework called MVC. It’s not new by web standards, it’s not even new by .net standards as the Monorail project had been alive for years before. Yet people were waiting for it. People were waiting for Microsoft to deliver it to them. There were already open source MVC web frameworks and an army of .net developers who could have collaborated to make them great but nothing much happened. It took a handful of developers at Microsoft to make their own framework and now MVC is the hot new thing.

But they did not:

This new web framework is better than the previous one and is quickly becoming the de-facto. Microsoft released the source. Someone created a github repository. This repository is being watched by 30 people and 5commits have been made to it. 5 commits! Why is this number so horrendously low? Because Microsoft don’t take patches. They’ll release a new version of MVC without anyone’s commits. Worse than that, everyone will start using their new version and the github repo will just start again.

The little community that exists for Microsoft wants to see success but Microsoft won’t let them.  Microsoft doesn’t believe in community, they may say they do but actions speak louder than words.  In the end, they are only hurting themselves.

Microsoft has been touting open source for a while now, not because they want to but because they think it will improve their bottom line.  Isn’t that what they’re all about?

Imagine if Microsoft would have opened up ASP.NET MVC completely and let the community create it?  Yes, I know they released the source, but that does not make the project open source.  Open source means they take contributions from the community and the code is created and owned by the community.  Big difference.

Why Leave .NET?

Well, for one, to make a statement.  If all .NET developers accept the status quo, nothing will change.  Secondly, to see what it looks like from outside the .NET world, how other developers live and breath.

You don’t have to actually leave the technology, maybe explore others and gain some experience.  The experience could make you a better developer and bring back something to .NET, possibly to help Microsoft become aware of a better way.

Where to go?

Well, almost anywhere else.  I like the Ruby community but I have heard Python has a great community.  I would bet almost any open source community will welcome you.

What’s the answer?

I developed C# code since it’s inception and wrote many lines of C++ using Microsoft Visual C++ before that.  These are really good technologies and we can’t just write them off.

I hear developer’s stories all the time, how they dropped .NET and moved to technology X.  The Ruby community itself contains many ex-.NET developers who have seen the light.  How can we make Microsoft see the light?  I don’t know we can, they need to be looking.

It comes down to this, Microsoft is losing without a community.  I am referring to the passionate community, like the 1600 who created Ruby on Rails.  I think as more and more developers gain experience and see how other communities function, they will leave and move on.  It won’t be overnight but it will happen unless Microsoft is willing to change.

So to answer the question, where are we without a community?  The answer is no where.

App Inventor for Android, VB Revisited

Google announced a new product today, App Inventor for Android.  This is a development environment for creating applications on Android, only you don’t need to be a developer, says Google:

You can build just about any app you can imagine with App Inventor. Often people begin by building games like WhackAMole or games that let you draw funny pictures on your friend’s faces. You can even make use of the phone’s sensors to move a ball through a maze based on tilting the phone.

But app building is not limited to simple games. You can also build apps that inform and educate. You can create a quiz app to help you and your classmates study for a test. With Android’s text-to-speech capabilities, you can even have the phone ask the questions aloud.

To use App Inventor, you do not need to be a developer. App Inventor requires NO programming knowledge. This is because instead of writing code, you visually design the way the app looks and use blocks to specify the app’s behavior.

Here is App Inventor in Action:


Is this the new Visual Basic?

I think we saw this before with Microsoft’s introduction of Visual Basic.  Visual Basic was an environment for just about anyone to create applications for Microsoft Windows.  Sure, it was used a TON and created a rich marketplace but it also created a lot of really bad software with really poor code bases which were hard to maintain.  I think a few readers probably still have their fair share of VB 6 code out in the wild.

I saw so many applications created by dragging-and-dropping controls on a form by business people, farm-hands and anyone else who cared to pick-up a mouse.  These applications were often toys created by mort.

Dave Winer of Scripting News, agrees this may not be the thing for the marketplace.  He certainly has seen technology come and go:

This idea of a once-and-for-all development tool is like the Divining Rod of the Olde Days. Perpetual Motion. The goose that laid the golden egg. The fountain of youth. Shangri-la. Bigfoot. The Loch Ness Monster. Cold fusion. The Singularity!

Jeff Jarvis and The Quark of Programming also offers an interesting perspective:

Will App Inventor yield lots of crappy apps? Of course, it will, just as Quark enabled sinful design and Blogger wasted bits. That is true of all such technologies that lower the barrier to entry to a former domain of priests. That’s precisely what the printing press did. As much as the web breaks down priesthoods, it created new ones. Developers are merely the latest. They say that mortals can’t do what they do. But what if they could? What if they could translate a thought not just into words and design but into action?

I imagine Marc Benioff of Salesforce.com going positively batshit over this, enabling businesses to create apps for, say, their sales teams to manage and share information about and with clients. I imagine small businesses using App Inventor to create apps like Chipotle’s that enable customers to make burrito orders before they arrive. I imagine teachers being able to make exercises and quizzes in apps (forget the electronic textbook; give me the electronic workbook!).

I think Jeff is right, there will be a lot of crappy applications created.  This could be a huge problem too.  Let’s say the Android app store skyrockets to 1,000,000 applications, how would you ever filter out the junk from the good?  It would be nearly impossible.  As it is, Apple has a pretty stick policy to filter apps fro there store and it is still hard to find an application never mind a good one.

On the other hand, from a business perspective, this is a potential great move by Google to create a ton of new applications for their store.  In one fell swoop they can have developers and non-developers creating applications for their store.  I speak from experience, it is not trivial to create iPhone applications, certainly not something a non-developer could easily tackle.

I would love to see a tool like this succeed, but I have seen others fail.  Why should App Inventor be any different?