Accidental Technologist

Musings about Entrepreneurship, Technology and Software Development

Overcoming the Mimemagic Fiasco

by Leave a Comment

I’m sure if you are a Ruby on Rails developer, you have heard about the fiasco that is Mimemagic.

It seems a component used in the Mimemagic gem was under a GPL license which was different than Mimemagic but because of how these licenses work, trickles down to users of Mimemagic. In this case the Ruby on Rails gem. If you’re interested in some of the background, the Ruby on Rails issue, Dependency on mimemagic 0.3.x no longer valid #41750, can fill you in.

The gem was yanked from Rubygems and because of this, the gem could no longer be installed so builds broke everywhere. Users received the message:

Your bundle is locked to mimemagic (0.3.5), but that version could not be found
in any of the sources listed in your Gemfile. If you haven’t changed sources,
that means the author of mimemagic (0.3.5) has removed it. You’ll need to update
your bundle to a version other than mimemagic (0.3.5) that hasn’t been removed
in order to install.

I have several clients needing a way to resolve this problem.

I came up with a couple different options.

Option 1 – Remove Dependency

After a bit of research I discovered that Mimemgic was used by the Marcel gem which is required by ActiveStorage. If the application did not use ActiveStorage then removing the dependency would solve the problem. If you use ActiveStorage, this is not an option.

Implementing this is pretty straightforward. Opening up your config/application.rb, you should see where Rails is required:

require "rails/all"

Instead of requiring all, just require what you need and remove the ActiveStorage dependency. This is what requiring rails/all includes:

active_record/railtie
active_storage/engine
action_controller/railtie
action_view/railtie
action_mailer/railtie
active_job/railtie
action_cable/engine
action_mailbox/engine
action_text/engine
rails/test_unit/railtie
sprockets/railtie

Just remove the second line and require each of these individually and you will be all set.

Option 2 – Change how the gem is included

Since the Mimemagic gem is open source and the code is in a git repo, we can identify the commit created when version 0.3.5 of the gem was released. Some commits that should solve the problem:

My projects have been using v0.3.5, I found the commit and added this to my Gemfile:

gem 'mimemagic', '0.3.5', git: 'https://github.com/mimemagicrb/mimemagic', ref: '01f92d8'

This is a short-term fix. I expect once the Rails team resolves the dependency, new versions of Rails will be released.

Redis::CommandError – MISCONF Redis is configured to save RDB snapshots

by Leave a Comment

I recently ran into a problem I hadn’t encountered before on my Mac. I was getting an error from Redis:

MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk. Commands that may modify the data set are disabled, because this instance is configured to report errors during writes if RDB snapshotting fails (stop-writes-on-bgsave-error option). Please check the Redis logs for details about the RDB error.

The fix is simple. From a terminal window, enter the Redis CLI. From a terminal window type:

redis-cli

Use the command:

config set stop-writes-on-bgsave-error no

You should get an OK response, then type exit to get out of the reds-cli.

Try again where you received the error and everything should be working as expected.

frozen_string_literal: the not so magical comment

by Leave a Comment

I have been working on a project over the past year for a client with a large Ruby on Rails application. The project has allowed me to learn more about how large Rails applications are developed and managed in the real-world. I’ve picked up some real gems to help scale Rails applications which I’ve used in some of the smaller applications I maintain.

One of the gems I learned about was the use of this magic comment:

# frozen_string_literal: true

This comment was added to every Ruby file in this company’s application, which included several microservices along with a large main application. The files numbered over 1,000.

I read up on the use of this magic comment use in other projects and the results seen. One in particular was Sidekiq from Mike Perham. Mike documented his use of the magic comment and how it improved his use and helped cleanup his code.

Ruby 2.3 introduced a very nice option: each Ruby file can opt into Strings as immutable, meaning all Strings within that file will automatically freeze, with a simple magic comment at the top of the file.

Mike includes some benchmarks that show improved memory footprint for Sidekiq. This was enough for me.

I had one relatively small application that had an API component I was optimizing (Rails 5.2.x and Ruby 2.6.6). I thought about the use of the magic comment and decided to give it a try. I added it to every Ruby file in the project and was included as part of the weekly deploy.

The deploy was done after hours one evening and I was ready to witness the magic.

Unfortunately, this magic did not happen. I noticed right away the response times of API calls were slower, not faster. Maybe I was seeing the effects of cold cache calls and things would improve overnight.

The next morning in fired up Skylight to check the response times from the API. To my surprise, the response times did not improve. Since the only change in this deploy was the use of magic comments, I decided to rollback the changes. This application runs on Heroku, rolling back with really easy and does not require reverting the code back and redeploying.

The result was noticed instantly:

IMG 1186

Response times drastically improved as you can see from the above graph.

At this point, I don’t have any reason to believe it’s not the magic comment that caused these longer response times.

During my brief research about the comment’s use, thinking there was more to using it than simply using it everywhere, I ran across a thread from the Ruby lang issue list. The threads title, Reconsider impact of frozen_string_literal on dynamic strings, I found interesting. Maybe this is a clue and dynamic strings being frozen in my application is part of the problem.

It sounds like there is a benefit to use this feature but not everywhere, the exact result I saw in performance graphs.

I am still exploring when to use the magic comment and when it shouldn’t be used. If you are using this throughout your applications it might be worth measuring with and without their use. Any others have more information to share, please reach out.

“Are you building a business or learning a stack?”

by Leave a Comment

Stefan cosma 0gO3 b 5m80 unsplash

I always love when someone posts something online saying they have an idea for an application and ask, “What stack should I use?”

It’s a fair question, but the answers usually start with “it depends”.

What is a “stack”?

The “stack”, for those unfamiliar with the term, refers to what programming language, frameworks, and database a project uses to get its work done.

Examples could be Ruby, Ruby on Rails and PostgreSQL, or JavaScript, NodeJS and MongoDB or maybe C#, ASP.NET Core, and SQL Server. All acceptable choices to build a web application.

Indie Hackers

There is a post on Indie Hackers, titled Tech Stack Suggestions. The original poster wants to build an MVP for his business idea and is soliciting opinions as to what stack to use.

He’s wondering about using a frontend framework like React and replies to someone suggesting he keep it simple:

Shouldn’t I be using a frontend framework in order to develop this? Backend Django + SQL has its advantages, in fact, the current project I’m working on in my organization is partially based on that. But, I’d like to implement things like GraphQL, NoSQL primarily to get a more hands-on on how they’d work in a real project. Would really appreciate your input.

Volkan Kaya replies with brilliant and practical advice:

Are you building a business or learning a stack?
If you want to learn a stack do it while being paid, not while starting a business.

Nick Haskins pragmatically suggests:

It’s really unfortunate that our industry has convinced you that you need to utilize tooling built for larger applications. On behalf of the industry, I apologize for that.
You do not need a front-end framework to develop a web app.
I’ve built multiple very large apps with Rails + Bootstrap. No ReactJS. No VueJS. No JS libs, just a sprinkle of jQuery.

These are examples of excellent and practical advice. The responses to the post include a lot of misguided answers as well.

Shiny Things

The thread from Indie Hackers sums up what it’s like to be a technologist who wants to build their software. You should give the thread a read.

It’s easy to want to build something and want to use the latest frameworks. It’s a great chance to learn something new. I’m all for learning; you should never stop. But, if your goal is to build a business and challenge yourself with new technology while doing it is not the best decision.

I’ve been guilty of trying to convince myself there’s some technology I need to use because applications that are created with it are faster or scale better. It’s not essential before you have that problem. People will try to justify using it anyway.

Don’t follow like sheep do…right off a cliff.

Love the One You’re With

It’s best to take a step back and breathe. Take stock in yourself and where your strengths lie. Ask yourself which programming language and framework you use most today? Which one do you make a living using today?

What is wrong with this combination that you wouldn’t want to use it to create your application? You want to make the right choice and not regret it later.

In its most basic form, people want to make an application and try to think too far ahead. Thoughts of scaling, finding developers, performance, and such things that don’t matter until later.

As a software entrepreneur, more appropriate questions you should be asking yourself are:

  • Am I solving a real problem?
  • Do I know people who will pay me money for my solution?

These are the questions to ask yourself now. Notice these have nothing to do with the stack you choose.

The simple answer to the stack question as a developer is to use the stack, you know, leading from idea to product in the shortest amount of time.

Why strap yourself with two hard problems? Toss the stack problem right out the window.

The best advice is to use the framework you are most comfortable with and will be the most efficient—the one you can get done what you want in the shortest amount of time. You’ll be happier and have something to show for it.

Don’t let people try to convince you their stack choice is better. They will have convincing arguments. None should sway you.

Exceptions

The above is my rule, but there can be exceptions.

As a Ruby on Rails developer, if I want to build a native Mac application, then I probably need to use Swift or Objective-C. Yes, I could use something like React Native; that’s not the point, so please don’t send me an email to the effect. If I don’t know the language, then I have given myself multiple challenges.

I’m sure there are other instances where I would need to learn something new, but the cases will be unusual when you’re deciding the application to build.

Conclusion

If you’re planning on starting a software business and you’re going to be the primary developer, then use the tools you know best. You are in a great position to get something out for your customers to see. Remember, they don’t care what your application is written in, only that it solves their problems.

11 Ruby on Rails Podcasts Worth Your Time

by Leave a Comment

Ruby on Rails

There is great value in the Ruby on Rails podcasts and screencasts we have available today. Some podcasts have gone away while others have appeared, and others have changed hosts. The title is a bit deceiving, I’m including some screencasts too.

I wrote up a list of Rails learning resources last week, which included some screencasts. I list those here to keep the resource consistent for folks finding this in the future.

Podcasts

I have far too many podcasts in the Overcast app on my iPhone, but these always get listened to first.

Remote Ruby – three developers, chat about Ruby on Rails, what they’re up to, and the community at large. Occasional guests.

Ruby on Rails Podcast – this podcast has been around a long time and has seen a few hosts. Episodes consist mainly of interviews with people in the industry or who use Rails for their jobs. I do really like the guests and topics, many are the cutting edge of what we are being exposed to in work today.

Ruby Rogues – a long time show featuring a panel of known people from the Rails community who discuss various topics with guests. Episodes are mainly a single timely topic the panel and guest discuss. Approaching 500 episodes.

Rails with Jason – interview-style show with Jason Sweet. Jason has some great guests on his show. If you can listen to only one, I’d try this one. If you can’t get enough of Jason, he has written some great articles.

Maintainable – hosted by Robby Russell, long-time Rails developer and founder of Planet Argon, a Rails development agency. From the Maintainable site:

On Maintainable, we speak with seasoned practitioners who have worked past the problems often associated with technical debt and legacy code. In each episode, our guests will share stories and outline tangible, real-world approaches to software challenges

Running in Production – a podcast about how folks are running various frameworks in a production environment and what it takes to do so. Not strictly talking about Rails, but there are a handful of episodes specifically dealing with challenges of running Rails in production.

The Bike Shed – discussion podcast, with two people from Thoughtbot. Much of the discussion encompasses issues the hosts face while working at the company. The episodes aren’t strictly Rails but cover topics many of us face in our day-to-day development work.

Screencasts

Drifting Ruby – created by Dave Kimura, also a long-time Rails developer. Dave is currently up to 227 episodes with episodes running from ~10 min to ~30 min. You might think these screencasts are probably like those from Go Rails, hardly. I think they complement each other very well. Even for topics, they are the same. I find the approaches very different. There are also free episodes, while others require a small monthly fee.

Go Rails – created by Chris Oliver, who is a great contributor to the community. At the time of this writing, Chris is up to 330 videos of varying lengths (~5 min to ~30 min) covering a full breadth of topics, including everything from Rails concepts to the inevitable problem you’ll face when creating Rails applications. There is also a forum that accompanies the videos were you can ask questions or answer some if you so choose. There are some free videos and a Pro plan you can pay to get the rest of the videos for a reasonable monthly fee. It’s a bargain for sure.

RubyTapas – created by Avdi Grimm, a long-time member of the Ruby community. From the RubyTapas website:

RubyTapas is for the busy Ruby or Rails developer who is ready to reach the next level of code mastery. Short, focused screencasts twice a week will introduce you to a wide variety of intermediate to advanced Ruby concepts and techniques, Object-Oriented design principles, testing practices, refactoring skills, and much more.

RailsCasts – created by Ryan Bates, these short screencasts were the original for the Ruby community focusing on Ruby on Rails. Ryan stepped away from making these a while ago, but many are still relevant today and free.

There’s so much great content in these resources. Whether you like to listen to podcasts or prefer to watch people code in a screencast, there’s plenty to learn. If you find other Ruby and Rails related podcasts, I’d love to know about them.

10 Ruby on Rails Learning Resources for 2020

by Leave a Comment

Even though Ruby on Rails has been around since 2004 as an open source project, I still get asked the best ways to learn the framework today.

It’s a fair question and answers do change as the Ruby on Rails landscape has evolved.

Beginner Level

You have little to no Ruby on Rails knowledge and are looking for resources to get you started out right.

  1. The Ruby on Rails Tutorial by Michael Hartl – now in it’s 6th edition and keeping up with Rails 6. This book and video options (20 hours) help many Rails developers get started. It’s written my Michael Hartl and comes in at 883 pages. It is a well-written tutorial.
  2. Ruby on Rails Guides – when talking about getting information from the source, this is the one to use. Each section contains the documentation on all the major parts of the Ruby on Rails framework broken up by function. A nice feature is users can select the version they are using, starting with the latest as of this writing, 6.0.2.1 all the way back to 2.3.
  3. Agile Web Development with Rails 6 – the Rails 6 version of the book which has gotten many Rails developers started, including myself. The Pragmatic Bookshelf offers a version back to Rails 4 so make sure you reference the correct version. This version will help with coming up to speed on the new Action Mailbox and Action Text. It offers source code for the project, which is a nice eCommerce application with practical value.
  4. Agile Web Development with Rails 5.1 – this version of the book is, you guessed it, targeting Rails 5.1. It contains the latest updates for 5.1 and offers source code for the project, which is a nice eCommerce application with practical value.

Intermediate Level

You’ve built a couple applications, know the beginner material and have a couple years of Rails experience. You are ready to move on to hone your skills.

One of the-best ways to learn, for me and I’m sure others, is by watching screencasts. The next two resources are they best out there.

  1. Go Rails Screencasts – created by Chris Oliver who is a great contributor to the community. At the time of this writing Chris is up to 330 videos of varying length (~5 min to ~30 min) covering a wide breath of topics, including everything from Rails concepts to the inevitable problem you’ll face when creating Rails applications. There is also a forum that accompanies the videos were you can ask questions or answer some if you so choose. There are some free videos and a Pro plan you can pay for to get the rest of the videos for a reasonable monthly fee. It’s a bargain for sure.
  2. Drifting Ruby Screencasts – created by Dave Kimura, also a long-time Rails developer. Dave is currently up to 227 episodes with episodes running from ~10 min to ~30 min. You might think these screencasts are probably like those from Go Rails, hardly. I think they complement each other very well. Even for topics they are the same, I find the approaches very different. There are also free episodes and others are paid for with a small monthly fee.
  3. Modern Front-End Development for Rails – when moving to recent versions of Rails, 5.1+, you have exposure to technologies such as JavaScript with the required tooling. This book takes you there and helps clear up some of the confusion that will certainly arise.

Advanced Level

You feel good about your Ruby on Rails skills and want to press on to even more advanced topics.

  1. Metaprogramming Ruby 2 – if you aren’t sure what metaprogramming is, it’s worth learning about and use it where it makes sense. It’s a technique that’s used in Rails and other Ruby frameworks and applications. It’s powerful. This book is the best resource I’ve found on the subject from just learning it to becoming proficient.
  2. Rails 5 Test Prescriptions – the Rails community is all about Test-Driven Development (TDD) and this book gives great coverage of the subject. It covers RSpec and mintiest. RSpec is probably the most popular testing tool for Rails. Mintiest is the default testing framework.
  3. Docker for Rails Developers – unless you’re living under a rock you have at least heard of Docker. This book is the only source I am aware of that directly helps Rails developers containerize their application. It is a great single source.

If you’re just starting out, a seasoned expert or somewhere in between. I think these are some of the best sources of guidance and knowledge available for our beloved Ruby on Rails.

If you think I forgot something, please leave a comment and let me know.

When You Can’t Check a Checkbox using Capybara

by Leave a Comment

In my last post I documented an issue I was having using Capybara on a client’s Ruby on Rails application with a ReactJS frontend.  I wanted to share another issue I came  across during my feature testing escapades.

The Problem

During my recent feature testing project I had a form which has a checkbox on it. The checkbox had a label with it. Did I mention this is a ReactJS frontend? I’m not sure if this is specific to ReactJS, but I suspect it isn’t. I think other frontend JavaScript frameworks may exhibit the same problem.

The Ruby code for my feature test is dead simple:

check “English”

That’s it. The test should run and when it finds the checkbox with a label of English, the checkbox should be checked. But, it doesn’t work. After many attempts at making this work and more Google searches than I can remember..I ended up at the Capybara mailing list.

The Solution

Thomas Walpole was kind enough to reply with his thoughts on the matter:

99.9% sure your checkbox isn’t actually visible on the screen.

What you’re describing as the “checkbox” is probably an image (probably added via CSS pseudo elements) being shown in place of the actual checkbox input element to ensure the same styling across different browsers.  If the checkbox has a label element correctly attached to it you can use `check(‘whatever’, allow_label_click: true)` – https://www.rubydoc.info/ github/jnicklas/capybara/ Capybara/Node/Actions#check- instance_method –  to tell Capybara to click the label element instead of the hidden checkbox input to toggle state.  If you don’t have an associated label then you’ll need to find whatever element is actually visible on the page and click it instead.

Changing my test to include this for the checkbox, worked perfectly.

check(“English", allow_label_click: true)

I hope someone finds this valuable and will save them some time and hair pulling.

Fixing StaleElementReferenceError When Using Capybara

by Leave a Comment

I’ve been spending an extended period of time lately writing Feature specs for a Ruby on Rails project using Capybara. Overall it’s been a fun project to work on but I had run into a few issues along the way. One in particular is the StaleElementReferenceError exception that get thrown when you are trying to find an element on your page that you know is there but Selenium can no longer find.

The error appears to be a timing issue between the page loaded in the browser, it being rendered and Capybara and Selenium trying to find the element. After much trial and error, along with many Google searches, I found no solutions that were reasonable. Even though many people had the same problem.

The solution was pretty simple actually. I added a rescue to my test and simply slept for 1 second and retry the failure. I admit, this seems like a hack but it works and that’s all I really cared about at the time. I see no ill effects from this, no measurable delay.

RSpec.feature "Listings", type: :feature, js: true do
  scenario "can edit common area - step 3" do
    begin
      post_a_space_step_1_with
      click_on 'Next'
      post_a_space_step_2_with
      click_on 'Next'
      click_on 'Edit Common Area'
      expect(page).to have_content "Step 1"
    rescue Selenium::WebDriver::Error::StaleElementReferenceError
      sleep 1
      retry
    end
  end
end

I’m certainly open to a better solution or an explanation as to why this might be bad. Please leave comments. Hopefully I help someone get around this really annoying problem.

Ruby on Rails Testing Resources

by Leave a Comment

When taking the plunge into Ruby on Rails it’s really easy to get carried away with learning all about the framework. It’s easy to learn the fundamentals and later realize the Rails community is a community of testers. It’s a strange world when you set out to learn about testing, TDD (test-driven development), BDD (behavior-driven development) and other acronyms and phrases relating to testing Ruby on Rails applications.

I decided to put together a short list of helpful resources to get started. If you have suggestions that would be useful to be added to this list, please add a comment or email me directly and I’ll update this post.

Books

  • Everyday Rails Testing with RSpec – this is a great, hands-on, roll-up your sleeves and get-to-work book. If you want to use RSpec on a daily basis, this book gives great advice on how to use RSpec in your day-to-day routine. It’s kept up-to-date with latest RSpec too.
  • Rails 5 Test Prescriptions – I use this book as a reference I often go to. It’s been updated to from previous versions to now Rails 5 and is a great tool to have on the shelf.
  • Effective Testing with RSpec 3 – if you decide you’d rather start without worrying about all the details around Rails you can start with learning RSpec with plain Ruby and help yourself. I’ve been through this one cover-to-cover and it’s a great tutorial.
  • The Minitest Cookbook – if you decide RSpec isn’t for you, this is probably the ultimate resource for Minitest. Well-written and kept up-to-date.

Podcasts

You can’t really learn testing from a podcast but you can learn how others approach the craft. The first is a podcast dedicated to testing Ruby applications. The rest is a list of a few episodes of podcasts that discussed testing.

I’ve been listening to The Ruby Testing Podcast and picked up some nice tidbits so far.

Training

I love Pluralsight.

Xavier Shay has long been involved in the Ruby community and well-known for discussions around testing. One of his best blog posts explains his approach to testing.

I’ve taken several courses on Udemy and they are one of my favorite places for training. The prices are low and there are many courses, so you have to do a bit of work to see which course is right for you but well worth the effort.

Ruby on Rails – Supporting SSL for PostgreSQL on Heroku

by Leave a Comment

For anyone who received this notice from Heroku:

Since 2016, all newly-provisioned Heroku Postgres databases have enforced the use of SSL to keep your data safe. However, one or more of your Postgres databases are running on legacy infrastructure, which does not enforce the use of SSL. In order to update your database to our security standards, and in response to potential impacts caused by Spectre and Meltdown, all databases – including those on legacy infrastructure – will be moved to our new Heroku PGX plans in a set of maintenances starting in March 2018 and concluding by April 2018.

What Do I Need to Do
In preparation for these maintenances, please check that your applications are using SSL to connect to your Postgres database and enable SSL connections if needed. Instructions on how to perform these steps are available in Dev Center.

If you’re using Rails 4.1+ there is a support article on the Heroku Dev Center, that helps clarify making updates to our database.yml file. This allows customizing some connection behavior to PostgreSQL.

Some parts of the database.yml file that cannot be changed include:

You cannot use the config/database.yml to set any values found in ENV[‘DATABASE_URL’]. This is a list of attributes you cannot change:

  • adapter
  • database
  • username
  • password
  • host
  • port

But, what can be changed include sslmode.

production:
 sslmode: require (disable|allow|prefer|require)
 pool: 15

I decided it would be helpful to reach out to Heroku to understand their guidance with regard to their notice. Their response:

If you’re using the pg gem, the default sslmode setting (and for libpq, the library that underpins it), is prefer this means that should the server have SSL support, it will be used when the connection is established. This means there should be no action required, though if you wish, it’s worth a test with spinning up a staging environment with a non-legacy Postgres instance.

It seems if you’re using Ruby on Rails with the pg gem, you should be OK doing nothing but with brownout period scheduled, it’s probably a good idea to test during one of those times.

Heroku Support also indicated setting the environment variable PGSSLMODE would also override the default behavior for sslmode used by libpq.

It seems this is a notice which doesn’t effect a majority of Heroku customers and is a necessary and worthwhile upgrade. Hopefully this helps others as the public information available for this from Heroku is minimal.