#ReplaceCodeSchoolsWithCommunity2016

My friend Pamela had some thoughts on code schools that she shared on Twitter today and I just have to echo them in every venue that I have. Code schools have long made me feel uncomfortable. It really started when a local code school started amping up representation at RailsBridge (free & community driven), obviously with an aim to drive RailsBridge grads to their boot camp. Considering the other TAs were there from companies that were interested in hiring, I thought it really sleazy that they were just using it to try to recruit more students. Add this in to women generally needing more validation before they feel comfortable calling themselves developers and that makes me feel even more uncomfortable.

I try to do my part by running PyLadies Boston and Pamela runs Rails Girls Atlanta. Both of these groups exist to provide a community for women to learn to code, with no charge. I’ve actually had multiple women from my group start applying to jobs as developers and I know the same goes with Rails Girls. Community can make a big different and we need to let women know that you don’t have to shell out $$$ to become a developer. The community will help!

PS - I have met so many wonderful and smart grads of code schools. This is not a knock on you! It’s a knock on this f’ed up industry.

A Bash Script For When You Need To Duplicate Host Keys Across Servers

Why would you need to do this? Let’s say you have three production nodes: server1.test.com, server2.test.com, server3.test.com. In general, your production application (at prod.test.com) point to server1.test.com. But OH NO you totally messed something up on that node and you need to easily failover to server2.test.com. Easiest way? If they all have matching server host keys, you can just point prod.test.com to server2’s ip and you won’t get any errors. Otherwise, you’ll get loads. I also added a noop (no operation) mode to this script so it could be run as ./move_keys.sh -n and only print out the lines that it would run.

This script is written to be run on the server you want to copy the keys from as your user, which means you have to have ssh access into each server. For each server in the list that is not the host, it backs up the /etc/ssh directory, rsyncs the keys over, restarts ssh, then removes the relevant lines for that server in the known_hosts file.

#!/bin/bash
# use noop mode by passing -n
NOOP=false
# parse the options
while getopts 'n' opt ; do
  case $opt in
    n) NOOP=true ;;
  esac
done

SSHC='/usr/bin/ssh'
HOST=`hostname -f`

# Params: $1, host to log into and copy ssh dir into backup dir
copyit() {
  if [ $NOOP = true ]
  then
    echo "$SSHC -A $1 'sudo cp -r /etc/ssh /etc/ssh_bak_$(date +%Y%m%d%H%M)'"
  else
    $SSHC -A "$1" "sudo cp -r /etc/ssh /etc/ssh_bak_$(date +%Y%m%d%H%M)"
  fi
}

# Params: $1: src path, $2: dest path
moveit () {
  if [ $NOOP = true ]
  then
    echo "sudo -E rsync -avz -e 'ssh -o StrictHostKeyChecking=no' --rsync-path='sudo rsync' $1 $2"
  else
    sudo -E rsync -avz -e 'ssh -o StrictHostKeyChecking=no' --rsync-path='sudo rsync' $1 $2
  fi
}

# Params: $1, host to log into
restartit() {
  if [ $NOOP = true ]
  then
    echo "$SSHC -A $1 'sudo service ssh restart'"
  else
    $SSHC -A "$1" "sudo service ssh restart"
  fi
}

# Parmas: $1: host to deploy keys to
removeknown() {
  if [ $NOOP = true ]
  then
    echo "sudo ssh-keygen -f '/root/.ssh/known_hosts' -R $1"
    echo ""
  else
    sudo ssh-keygen -f "/root/.ssh/known_hosts" -R $1
  fi
}

# Params: $1: host to deploy keys to
movekeys () {
  copyit "$1"
  moveit "/etc/ssh/ssh_host_key" "$USER@$1:/etc/ssh/"
  moveit "/etc/ssh/ssh_host_key.pub" "$USER@$1:/etc/ssh/"
  restartit  "$USER@$1"
  removeknown "$1"
}


set -e

declare -a Servers
Servers[0]='server1.test.com'
Servers[1]='server2.test.com'
Servers[2]='server3.test.com'

Servers_minus_host=( ${Servers[@]/$HOST*/} )

for server in "${Servers_minus_host[@]}"
do
  movekeys $server
done

Some Thoughts On Soylent

jenlovestheweb:

I’ve gotten a variety of reactions when I’ve told people that I drink Soylent. Usually it’s gentle mocking or ‘BUT WHAT DOES IT TASTE LIKE???’. Both of these reactions make sense… on some level, I think I’m a bit crazy for drinking it. On the other hand, it saves me so much time. I currently drink Soylent for only about 8-10 meals per week, generally breakfast and lunch during the work week. Pre-Soylent, I spent hours on Sunday prepping my meals for the week or was constantly eating leftovers; either way, I was eating the same thing almost every day. Or even worse, I went out to lunch, spent $10, and ended up getting something that wasn’t very healthy for me. Soylent costs about $1.92/500kcal meal - plus one point for cheap! It’s also pretty well balanced nutritionally, so I know I’m getting a decent amount of nutrients per meal. The other benefit is it’s bland taste… it’s hard to describe, but I just don’t even think about it. Since I don’t think about it, I don’t get sick of it. Overall, I like it and I’ll probably keep ordering it for the foreseeable future.

Relevant to all of you programmers out there considering switching to the nerd smoothie.

Sanitize Text in Perl

One of the projects I’m working on is in Perl. Having never touched Perl before, this is a bit of a new adventure! Today I had to sanitize a field being passed by a url. This took me a few google searches before I found the right answer, so I’m going to write down the solution really quick here so I don’t forget it.

Here’s the code:

# assuming $message is your user-inputed data
use HTML::Entities;
my $encoded_message = HTML::Entities::encode($message);

Then you can print your $encoded_message out in html and you will not be vulnerable to XSS attacks.

I literally just looked at Perl for the first time today, so any tips are welcome!

When Bad Things Happen to Good Jobs

No job is perfect. This post is going to cover some types of those imperfections (from personal experience) and what you can do to fix it (or at least make your job a bit better). One piece of advice applicable across the board: if something seems off, document it. Get a weird IM? Save it. Even if something is good now, you’ve got to protect yourself in case the situation gets worse.

1. Retaliation

How to spot it

Retaliation is not acceptable at most companies. However, most people are aware of that, so the way they retaliate can be more nefarious and more difficult to prove. This is one situation in which it is definitely critical to document everything. The two things to look out for are: 1) did a triggering event happen? Example: did someone get fired and could you be blamed in any way? 2) has one of your coworkers’ behavior towards you changed dramatically? Example: did you formerly have a good relationship and now they are criticizing you and complaining about you?

How to fix it

If this person is not your boss, I would first talk to your boss. Unless you have direct proof, it doesn’t serve you to be acussatory, but you can at least clear your name. You can possibly say that you think the retaliator seems to have an issue with you and ask for advice on how to handle it, or ask if you should change teams or departments.

If this person is your boss, go directly to HR. Again, unless you have direct proof, I would stay away from harsh accusations. Say something like “I’m not sure what happened, but ever since [triggering event], Joe has treated me differently and I’m not sure it’s healthy for me to continue working under him.” Ask for advice, see if you can transfer to a different manager, and possibly start updating your resume. Do not be afraid to talk to HR because that is not a fireable offense. However, if it doesn’t go very well, it’s probably not the type of company you want to work for.

2. Boss Is Not a Good Manager

How to spot it

There are so many versions of this. My favorite is the super micro-manager. Or the “never there and you don’t know what to do” manager. Or the “for some reason wants to make sure you don’t grow professionally” manager. All are bad, just on different levels. Depending on how you work, the micro-manager or never-there manager might not be too bad. Or not.

How to fix it

When you have issues like this with your boss, I would go to HR directly, especially for the second two. If you aren’t getting support (because they are never there or are denying you the opportunity for advancement), that has a huge impact on your career and your productivity at the company. Most HR professionals will listen and try to find a solution. This happened to me and I was moved to a different boss within a few weeks. For the micro-manager, I would actually first try to talk to your boss directly, if they are approachable. If you frame it as you wanting the opportunity for more independence and “larger” projects, the conversation could go quite well. No one likes to be told that they are doing something wrong, so make sure it is about what you can do, not what they are doing wrong.

3. Boss Dislikes You

How to spot it

This one can be sorta hard to tell. I think that you can know, but it’s definitely hard to prove and your boss would probably never admit it. If they admit it, then I would go to HR. Otherwise…

How to fix it

If your work environment is good otherwise, I would stay and keep doing good work. Your boss may not be a good reference, but if you keep submitting excellent work, your coworkers will be. I had an experience like this (and my suspicions were confirmed after I left). I still had great work experience to add to my resume, great coworkers to add to my network, and just a man who I know I don’t want to work for again.

If your boss’ dislike of you is really affecting your work environment, I would polish up that resume and leave as soon as possible. If you have proof, take it to HR.

4. Coworkers Are Inaccessible/No Help

How to spot it

This can be a fairly common problem when you are a junior developer or a new employee. It can be especially common if a lot of people work from home and turn their IM off. This can be especially hard if you need to get a particular piece of information and the person everyone points you to is in their “dev cave”.

How to fix it

If you are in an office full of people like this, get out now. Teamwork is an important part of growing as an engineer and obviously you are in an environment that doesn’t value it. Gross.

However, this is unlikely. Most of the time it will be a small handful of people. Search out the people who are available and willing to help you and reach out to them. If you have a decent boss, request a one-on-one and discuss the situation with them. Most bosses want their team to communicate and will not be pleased if they find that quite a few of their engineers are radio silent most days.

5. Total Disrespect

How to spot it

This is luckily an uncommon problem and also can be really easy to spot. However, in my experience, it was the problem that I felt most like I could fix and yet was totally unfixable. If you are constantly getting second-guessed, but nothing you have said has been false or even inaccurate, that is a form of disrespect. If your superiors are saying negative things about you to other employees of the company, that is a form of disrespect.

Those are a bit more outright, but disrespect can also be less blatent and take the form of microagressions. For example, if your coworkers, especially your higher ups, are smiling and nodding at you, then completely ignoring your suggestions and doing the opposite of what they said they’d do (what you recommended), that can be a microagression and a form of disrespect. This can be hard to spot because it usually doesn’t happen all at once, but if you are stripped of the ability to do your job effectively, that is also a form of disrespect.

How to fix it

You probably can’t. Definitely document everything. In a situation like this, you can’t be sure what will happen. Save all your emails, save all your IMs, and, if possible (and with permission of the people in the meeting), record conversations. If this is actually happening with a company large enough to have HR, definitely go to HR. However, that is fairly unlikely. You can try to have conversations with the people or person who are/is being disrespectful, but you definitely have to be ready to quit. I’ve had one job on this level. I kept bringing things up and even coming up with solutions in meetings that were agreed upon, would think things would improve, and they never did. In the end, that was the only job I walked out on. As a human being, you should not be treated like that, but, if you are a programmer, the job market is good enough right now that you absolutely do not have to be treated like that.

Job Search Retrospective

Wow what a month. I’ve been interviewing for a little over a month now. Two weeks ago, I quit my job (post on that mess coming up in a a few more months) and somehow got a job offer the same day. This has definitely been my shortest job search (time when I submitted my first application to first offer), so I wanted to share what I think has been different.

Finding Potential Employers - Recruiters & Networking

So first things first… I may run a fairly large meetup group, but I’m actually pretty bad at networking. If you are a networker and can go to a ton of meetups and talk to people, that’s the best way, hands down. First interview I got was because I went up to someone who I knew was hiring at a conference and talked to them.

If you aren’t much of a networker (like myself), working with recruiters is the way to go. If you are in one of the areas they serve, I have had really good luck with Talener, their Boston office in particular. They have put me in front of some great companies and I’ve had a ton of in person interviews as a result. 

Many people say that you don’t get anywhere from submitting your resume just from the company’s website. NOT THE CASE! I got my last job like that and, during this search, I got a great interview by doing that.

Now that the interviews are lined up…

Interviews!

The biggest change is obviously experience. Time definitely makes each round of interviews a bit easier! There were a few other things that made it a bit easier though. 

The first was inspired by Julie Pagano’s own job search retrospective. I’ve never been very good at asking a bunch of questions, but I looked at the questions she and Julia Evans asked and took a bunch of my favorites. I had a notebook and wrote out all the questions and all the answers. This helped immensely because, not only did I find out more about the companies and how they worked, but it also helped turn the tables a bit to have me interviewing them. 

The second was that I finally actually thought of a good answer for ‘Tell us about an interesting/challenging problem that you have solved and how you did it’. I was asked variations of that question at almost every interview and this is the first year I felt I had something to talk about. I was just talking to a junior dev about this question and, really, anyone who has solved any problem with code should have an answer to this question. If you have a project, you must have created it to solve a problem. If you had a bug in your project and fixed it, there’s your problem. Sure, it’s not the most interesting, but you will get better and better answers as you gain more experience, and employers recognize that.

And the one thing you cannot forget in tech interviews, WHITEBOARDING. Three years into this and whiteboarding is still nerve-wracking and I still feel like I’m awful at it. One thing I have learned: how the interviewers interact with you while whiteboarding tells you a lot how they would interact with you when actually problem solving on that job. If someone is making you feel stupid while whiteboarding, then you probably don’t want to work with that person. On the flip side, If you are having a bit of trouble and the person interviewing you is actually helpful, then that is the type of person that you want to work with.

Overview

Interviewing is tough. No matter what level you’re at, you’ll get people who will tell you don’t have enough experience for the job. Don’t sweat it. It just means it’s not a good fit… you’ll find a good fit somewhere else :D

Chosen & Multiselect

I’ve been doing a bit of work jazzing up our multiple select boxes. These plugins are going to be familiar to most people, but if you haven’t looked into both of these, you should. I had previously been using jQuery multiselect in my current app but have recently switched to Chosen to make use of it’s search functionality. Here are the two that I’m going to highlight:

1) Chosen

Chosen gives you the ability to do a type-ahead search. Instead of scrolling down through a long list, I can start typing and see what options match. Particularly helpful in dealing with food… if I don’t like almonds, it’s nice to see that almonds, almond milk, and almond flour could all potentially be ingredients.

2) jQuery multiselect

jQuery multiselect does require the scroll down list. You can use a separate plugin for search capabilities, but, by default, it doesn’t include it. However, if you have a shorter list, this is a very nice way to let the user select and deselect items.

No posts :(

And why? Because I’ve been pretty stressed and haven’t been doing my best work lately. I took a few days off to spend a few days in the mountains in Vermont. Relaxing, but the stress came back as soon as I returned. I promise I will post as soon as I actually have something to say.

One the upside, I have been exercising my stress away, going to Crossfit most mornings and then running after that about three days out of the week.

Rails Forms - Basic Errors

This is going to be short because I’ve been crazy busy, but nothing really worth blogging about. Two things about forms in Rails:

1) You can’t nest them within a table. Let’s say you have this bit of code in your ERB:

<tr>
  <%= form_for TestClass.new do |f| %>
    <td>
      <%= f.text_field :test_field %>
    </td>
    <td>
      <%= f.submit 'Create' %>
    </td> 
  <% end %>
</tr>

What this results in is this:

<tr>
  <form action='/test_class'></form>
  <td>
    <input name='test_class[test_field]' type='text'>
  </td>
  <td>
    <input value='Create' type='submit'>
  </td>
</tr>

The form ends up closing before it actually ends, which means that it won’t actually submit properly. Bizarrely enough, sometimes it works… but it won’t work consistently. Lesson: don’t nest forms in tables. Haven’t tried it, but there’s a possibility that with HTML5, if you define the form outside of the <table&gth; element, then it will work.

2) You cannot instantiate a form in one div and then end it in another. This may seem obvious, but I see this a lot, especially with juniors. Example:

<div>
  <%= form_for TestClass.new do |f| %>
    <%= f.text_field :test_field %>
</div>
<div>
  <%= f.text_field :num_ponies %>
  <%= f.submit 'Create' %>
<% end %>
</div>

Results in this:

<div>
  <form action='/test_class'></form>
  <input name='test_class[test_field]' type='text'>
</div>
<div>
  <input name='test_class[num_ponies]' type='text'>
  <input value='Create' type='submit'>
</div>

No bueno. Never nest anything across multiple elements. HTML does not appreciate it.

Reason for no posts for a week...

I was in Portland and Seattle over this past week on vacation. There was lots of this:

Promo Codes in Rails

When I got tasked with adding promo codes in our Rails apps, the first thing that surprised me was how few posts there were on how to do it. So here I am to fill that gap.

First! We have to create a promo codes model:

class CreatePromoCodes< ActiveRecord::Migration
  def change
    create_table :promo_codes do |t|
      t.string :code
      t.decimal :amount
      t.text :purpose

      t.timestamps
    end
  end
end

And the model:

class PromoCode < ActiveRecord::Base
  # ...
  has_and_belongs_to_many :users
  # ...
end

Then we should add some fields to a user to reference promo codes. For our case, we want to know what promo codes they have used in the past, how much credit they have, and what user referred them.

class AddFieldsToUser < ActiveRecord::Migration
  def change
    add_column :users, :credits, :decimal, default: 0.0
    add_column :users, :referring_user_id, :integer
  end
end

And the updates to the user model:

class User < ActiveRecord::Base
  # ...
  belongs_to :referring_user, class_name: "User", foreign_key: "referring_user_id"
  has_and_belongs_to_many :promo_codes
  # ...
end

Plus one final migration to allow the has_and_belong_to_many to work:

class CreatePromoCodeUser < ActiveRecord::Migration
  def change
    create_table :promo_codes_users, id: false do |t|
      t.belongs_to :user, index: true
      t.belongs_to :promo_code, index: true
      t.timestamps
    end
  end
end

We also have a PromoCodesController. This isn't really doing anything unique, we just need it so we can allow admins to create promo codes.

class PromoCodesController < AdminController
  before_action :set_promo_code, only: [:update, :destroy]

  respond_to :html, only: [:index]
  respond_to :json, except: [:index]

  def index
    @promo_codes = PromoCode.all
    respond_with(@promo_codes)
  end

  def create
    @promo_code = PromoCode.new promo_code_params
    @promo_code.save
  end

  def update
    @promo_code.update(promo_code_params)
  end

  def destroy
    @promo_code.destroy
    render nothing: true, status: 204
  end

  private
    def set_promo_code
      @promo_code = PromoCode.find(params[:id])
    end

    def promo_code_params
      params.require(:promo_code).permit(:code, :amount, :active, :purpose)
    end
end

We then added a route that was specific to adding a promo code (within the user controller). We post to this route whenever a user adds a promo code:

def add_promo_code
  user = User.find(params[:user_id])
  # We decided the easiest way to do referring users
  # was to have codes that looked like this: MYCODE-265,
  # with 265 being the id of the referring user.
  # Not necessarily teh best solution for everyone!
    promo_code, referring_user = params[:promo_code].split('-')
    code = PromoCode.find_by_code(promo_code)
    if code
      if user.promo_codes.include?(promo_code)
        message = 'This promo code has already been used.'
      else
        user.promo_codes << code
        user.credits += code.amount
        if referring_user && user.referring_user_id.nil? && User.find_by_id(referring_user).present?
          user.referring_user_id = referring_user
        end
        if user.save
          message = 'Promo code successfully used!'
        else
          message = 'Error using promo code. Please try again or contact customer service.'
        end
      end
    else
      message = 'Not a valid promo code.'
    end
  respond_to do |format|
    format.js {  flash[:notice] = message }
  end
end

Then, whenever we are getting ready to charge a user, we add code like this:

if @user.credits > 0.0
  if @user.credits >= @price
    @meal_plan.price = 0.0
    @user.credits = @user.credits - @price
  else
    @meal_plan.price -= @user.credits
    @user.credits = 0.0
  end
end

What do referring users get? For us, they get a credit when the person they referred makes their first purchase:

def charge_user(user)
  if user.first_charge? && user.referring_user
    user.referring_user.credits += PromoCode.find_by_code(@user.promo_codes.first).amount
    user.referring_user.save
  end
end

There are a few other smaller things, but that was the bulk. If anyone has any feedback or something they would do differently, I'd love to see it in the comments!

ActionMailer & Inline Attachments

This might have been a problem that arose from not looking very closely at ActionMailer documentation, but on Saturday, we did a deploy that ended up sending our customers emails with a ton of attachments.

So what happened? We were adding the inline attachments in a before_filter. That was fine. Except that we were adding ALL the inline attachments in a before filter, not just the ones that were needed for every email (say our logo). What we actually wanted to do is add the email specific attachments to just the individual methods within our UserMailer. So if our welcome email includes a certain image, we add it using inline attachments right in the method, like so:

class UserMailer < ActionMailer::Base
  before_filter :inline_attachments

  def inline_attachments
    attachments.inline['logo.png'] = File.read('app/assets/images/mailers/logo.png').force_encoding('utf-8').encode
  end
  
  def welcome_email(user)
    attachments.inline['image.png'] = File.read('app/assets/images/mailers/image.png').force_encoding('utf-8').encode
    @user = user
    mail(subject: "Welcome!", to: @user.email, from: "admin@test.com")
  end
end

More Favorite Gems

Here are another three of my favorite gems:

1. chronic - Chronic makes me want to do a happy dance almost every time I use it. It basically take text and finds a way to parse it as a date (if possible). Ever wanted to grab ‘next tuesday’? Chronic can do that for you.
2. pg_search - Hadn’t used it before, but then I had a very intense desire to rid my project of Solr (UGH THE WORST) and this gem did the trick. It makes it super easy to add full text search to your website if you have a postgres database.
3. jbuilder - I’ve been trying to make any new service respond with JSON (ahem, JSON API) and jbuilder is the perfect tool for creating those responses. Definitely makes my life easier :D

Got any gems that you love?

ActiveRecord Joins

Random thing I missed while turning all those selects into regular queries using joins: if you use the names of associations to join tables (ex. recipes.joins(:ingredients)), it creates an INNER join. Why is this important? Here’s my example:

I have recipes with ingredients, and each ingredient might have a subingredient (gluten, soy, etc). Every recipe has ingredients, so an inner join is fine there. However, every recipe does not have subingredients. Since I was doing a recipes.joins(ingredients: :subingredients), I was only getting recipes that did not have subingredients... definitely not what I wanted. Here’s the code:

BEFORE:
Code

Recipe.joins(ingredients: :subingredients).joins(:ingredients)

Resulting SQL

SELECT "recipes".* FROM "recipes" INNER JOIN "ingredient_measurements" ON "ingredient_measurements"."recipe_id" = "recipes"."id" INNER JOIN "ingredients" ON "ingredients"."id" = "ingredient_measurements"."ingredient_id" INNER JOIN "ingredients_subingredients" ON "ingredients_subingredients"."ingredient_id" = "ingredients"."id" INNER JOIN "subingredients" ON "subingredients"."id" = "ingredients_subingredients"."subingredient_id"

AFTER:
Code

Recipe.joins(:ingredients).joins('LEFT JOIN "ingredients_subingredients" ON "ingredients_subingredients"."ingredient_id" = "ingredients"."id" LEFT JOIN "subingredients" ON "subingredients"."id" = "ingredients_subingredients"."subingredient_id"')

Resulting SQL

SELECT "recipes".* FROM "recipes" INNER JOIN "ingredient_measurements" ON "ingredient_measurements"."recipe_id" = "recipes"."id" INNER JOIN "ingredients" ON "ingredients"."id" = "ingredient_measurements"."ingredient_id" LEFT JOIN "ingredients_subingredients" ON "ingredients_subingredients"."ingredient_id" = "ingredients"."id" LEFT JOIN "subingredients" ON "subingredients"."id" = "ingredients_subingredients"."subingredient_id"

UPDATE: As Darren (manvsmachine) mentions, you can also use .eager_load to create LEFT OUTER joins. However, that does give you a different query. It seems to have the same-ish results, but the SQL output is slightly different. It looks something like this:

Code

Recipe.eager_load(:subingredients)

Resulting SQL

SELECT "recipes"."id" AS t0_r0, "recipes"."name" AS t0_r1, "recipes"."instructions" AS t0_r2, ... "recipes"."image" AS t0_r14, "subingredients"."id" AS t1_r0, "subingredients"."name" AS t1_r1, "subingredients"."created_at" AS t1_r2, "subingredients"."updated_at" AS t1_r3 FROM "recipes" LEFT OUTER JOIN "ingredient_measurements" ON "ingredient_measurements"."recipe_id" = "recipes"."id" LEFT OUTER JOIN "ingredients" ON "ingredients"."id" = "ingredient_measurements"."ingredient_id" LEFT OUTER JOIN "ingredients_subingredients" ON "ingredients_subingredients"."ingredient_id" = "ingredients"."id" LEFT OUTER JOIN "subingredients" ON "subingredients"."id" = "ingredients_subingredients"."subingredient_id"

Optimizing Your Rails App

NOTE: This is only a short post on one thing that I did that drastically reduced memory leaks and helped the performance of my app. if you want to go more in depth, I would recommend Alexander Dymo’s Ruby Performance Optimization.

About a month ago, I became the lead (and only dev) at a small startup. I emphasize small because that’s important to what comes later. Soon after I started, I was seeing a load of Heroku R14 errors... to the tune of 200+ per day. At first I just paid for one more dyno... then another dyno. When I got to seven, I realized that I needed to do something else and fast, because we couldn’t keep upping the dynos every time we got a little bit more traffic. I was also still seeing a TON of R14 errors. Again, I’ll emphasize small, because we really don’t have the large user base that would be actually causing us problems like this. We were just having massive memory leaks. So, time to dig into the code!

The main issue seemed to be a function that saved all recipes that a user could eat (i.e.: they weren’t allergic, didn’t contain any of their dislikes, etc). We were originally determining this by doing something like this:

recipes = Recipe.all
recipes.reject! do | recipe |
  recipe.contains_allergens?
end
recipes.reject! do | recipe |
  recipe.contains_dislikes?
end
# and so on...

So what's the better, safer way to do this? Just build a query! ActiveRecord only runs a query when you perform an action on it, so I could build the whole query and then have my only action be saving it to User.recipes. Example:

user_recipes = Recipe.joins(:ingredients)
user_recipes = user_recipes.where('ingredients.id not in (?)', allergies.map(&:id))
user_recipes = user_recipes.where('ingredients.id not in (?)', dislikes.map(&:id))
# and so on...

This is a gross simplification of the actual code. I know that seems like a silly trivial change, but, no joke, it sped our tests alone up by 4 minutes. Why? Because previously, when we were doing all the rejects, Ruby was saving a new variable in each iteration. Also, while I forgot to test my hypothesis, I'm pretty sure that since recipes was an attribute of User and we were assigning to a 'variable' recipes within the User model, that it was actually saving during each call... also super memory intensive.

So what can you do? If you have some code in your app that you think is slow, you can test it using this benchmark script. If you have that added to your lib directory, just call it like this:

Measure.run do
  Recipe.last #whatever code you want to test
end

The output will look like this:

{"2.2.2":{"gc":"enabled","time":0.04,"gc_count":0,"memory":"0M"}}

To translate:

{ruby-version: {"gc": are_garbage_collectors_enabled?, "time": time_it_took_to_run, "gc_count": total_num_of_garbage_collectors, "memory": amount_of_memory_used }}

On Interviewing Developers

Just read this excellent article by Eric Elliot, Why Hiring Is So Hard In Tech. Here’s my favorite part:

Interviewing

These don’t work:

• Puzzles and riddles

• Whiteboard code tests

• Big O notation quizzes

• Detailed quizzes about the mystery corners of the language

These work great (in order of value):

• Pair program with candidate on an actual issue from your ticket queue (let the candidate drive)

• Code samples / OSS contributions

• PAID Sample project assignment (err on the side of paying fairly — say $100+/hour for estimated completion time — if the problem should require 2 hours to complete, offer $200)

• Review past work / portfolio

• Read candidate blog, publications, watch candidate talks

• Ask candidate for input on a real problem you’re currently working on

• Ask specific questions about software problems and solutions from candidate’s resume
  

IMHO puzzles and riddles as part of coding tests are the WORST. First off, no one is thinking their best during an interview. Second, solving riddles is (bizarrely enough) not a regular part of a dev job. Neither is whiteboarding (at least not like it’s done in interviews) nor chats about Big O notation. If you are interviewing developers you should ask questions that relate to what they actually might do. Recently, when I’ve been talking to candidates, I have been talking about actually problems that we have and it’s definitely helped me judge how great a candidate will be by their suggestions.

If you are looking for a job as a developer or hiring one, you should absolutely read this article.

Stop Undervaluing Yourself

TL;DR - Almost all interviews suck, don’t undervalue yourself because your interviews aren’t going that great.

I had a talk with one of the women in my programming group last night. She has a masters in CS, almost 2 years of work experience, and was still referring to herself as a junior developer. I spent about 30 minutes just repeating to her that she was not junior, just a regular developer… who does have experience! And education!

What prompted this total downturn in self-confidence? Interviews. Unless you are at the top of your game (in that case, good for you!), programming interviews are some of the most stressful things ever. More so than most other jobs, a rejection usually ends up sounding like they just don’t think you are smart enough, versus not a good fit. During my last round of interviews, right before I got a job, I had a bad run and starting feeling like I was in the wrong industry, stupid, and that no one would ever hire me again. On a normal day? Almost never feel like that. But both times I’ve done extensive interviewing, that is always the end result.

Why do I bring this up? Because if you are interviewing right now, don’t let the interviewers get you down. It’s not your intelligence or talent that they are rejecting. No one can really get to know you during an interview. At some point, you’ll go into an interview and they will ask only questions you can answer (or can figure out eventually) and that will be the place that is a good fit for you. Not the place that asks you weird logic puzzles until you are so mentally exhausted you can only babble on about how ‘code is nice’. Interviews aren’t just for the company… you are interviewing them. If the interview is super stressful, then it’s probably not the right place for you… but you will find that right place!

Nested loops

Is it possible to do nested loops? Absolutely! Should you seriously reconsider your code and make sure there’s not another way to do it if you are thinking of using nested loops? ABSOLUTELY! Sometimes you have no choice… but more often, you just need to sit back and think about a better way to do what you want. Here’s a bit of an example:

I work at a nutrition company. We have meal plans, meal plans have recipes, recipes have ingredient measurements, and ingredient measurements have ingredients. In one part of the app, we were trying to consolidate all the ingredients in a meal plan and their measurements. Originally, that involved something like this:

1. Get all the recipes
2. Get all the ingredient measurements in that recipe
3. Get all the ingredients
4. For each ingredient, get all the ingredient measurements, then loop through those and consolidate

Easier and less memory-intensive way of doing this:

1. Get all the recipes
2. Get all the ingredient measurements in that recipe
3. Loop through those once and tally them up using a hash, so your end result looks like this:

   {'1': [[1, 'cup'], [2, 'ounces']]}
   

Seriously, you should have a staging environment

 Having a good staging environment for a production app is vital. Ideally, it is almost identical to production, with the exception being test credentials if using a service like Stripe. When I came in, we had a staging environment. Ok, great! Except that we had no data in it, so we couldn’t actually test anything but the most basic interactions. It was somewhere between very difficult to impossible to test if bugs had been fixed. It was also using the production environment variable, so it was actually indistinguishable from production… but luckily, we were using test Stripe keys!

How we fixed it (in our Rails app):

1. Copied production.rb to staging.rb and replaced   config.action_mailer.smtp_settings with the settings from Mailtrap. If you aren’t up for Mailtrap (it’s pretty rad, I would definitely recommend it), you can also use just plain old gmail.
2. 2) If you are using the rails_12factor gem, don’t forget to add it to the staging environment in the Gemfile, like so:

   group :staging, :production do
     gem 'rails_12factor'
   end

   If you don’t do this, you will (like me) end up with your staging server being unable to find any of your assets and getting a ton of 404s in your console. Whomp whomp.
3. Set a cron job to regularly grab data from production. Here’s an example of how we are doing it:

   heroku maintenance:on --app STAGING_APP_NAME
   heroku pg:backups capture --app PROD_APP_NAME
   heroku pg:backups restore `heroku pg:backups public-url --app PROD_APP_NAME | cat` DB_NAME --app STAGING_APP_NAME --confirm STAGING_APP_NAME
   heroku run rake db:sanitize_passwords --app STAGING_APPNAME
   heroku run rake db:migrate --app STAGING_APP_NAME
   heroku maintenance:off --app STAGING_APP_NAME 
   

   where DB_NAME is something like HEROKU_POSTGRESQL_JADE. This type of script is only relevant if using heroku and postgres, but you can do something similar if using other providers/databases. This post goes over a different way to do it if you are using mysql. ‘rake db:sanitize_passwords’ would be a task of your own creation to sanitize all the passwords of the users (very important) and also (less important, but still good for privacy) remove identifying data.
4. Add staging to your database.yml. This should basically be a copy of production, but the database name should be something like #{product}_staging.

I’ve probably missed a few small things, but after that, basically just look for 'production’ in your app and add matching bits for 'staging’.

UPDATE: I did miss something after all. One of my former coworkers brought this to my attention: '3a) Rake task to sanitize/de-identify data, reset passwords?’. In the script I’m actually running, I am updating the credit card information so each customer just has a Stripe test account with a atest card instead of linked to their actual account, but I definitely did forget to reset passwords. Another reason this is useful is that, if you reset all your staging passwords to the same thing, you can login as any user on your test environment to debug any issues they may be having.