
2017 was a big year of growth and change for Olio Apps!

Our small team grew from four to seven people, and it was the first full year in our new office - having moved in October of 2016.

The last two and a half years have been a transition from solo to duo, to many person team.

In many ways, we’re becoming more and more of a “real” company.

We created a handbook, where we document The Olio Way to build software, our on-boarding process, and our standard policies and procedures. Everyone on the team contributes to improving it and making it a useful resource.

All employees now receive the full range of health benefits, a 401k plan, company equipment, funds for education, plus the all important free coffee and snacks. Another key benefit is technical mentorship. All employees have weekly one on ones, and we track annual goals for career development and growth.

When coding, we often work in pairs, which helps smash through hard problems, rather than try to solo-hero our way through it. We’re also big believers in peer review - all code goes through a pull request process where at least one other person reads through it, and either signs off or requests changes to fix or improve things.

We started a monthly lightning talk series this year. Anyone can give a short 5 minute presentation to the team on whatever they like. They’re generally about technical topics we’re learning, or the mechanics of a current project. The act of writing, and creating a short presentation really helps solidify knowledge, with the bonus of spreading it to all members of the team.

We attended two conferences this year, Gophercon in Denver and Chain React here in Portland. A few of us also attended a one day class on the basics of Sketch.

We even had our first official holiday party with our friends and families, complete with food, drinks, bowling, table tennis and video games. It was pretty sweet.

We set up this blog half way through 2017, and posted a few articles last year. It was a great start, but we’re planning to ramp up the frequency in 2018.

Shoutbase, our approach to time tracking, received numerous improvements, including new teams and goals features, as well as a redesign and new logo. Almost weekly, we do a one-day team “agile swarm” on Shoutbase where we knock out some new feature or smash some bugs.

We shipped production code to client projects in at least 6 languages - backend services in Go, Java, Python, and Ruby, frontends in JavaScript, and now TypeScript.

TypeScript is the latest addition to our stack and a huge boon to productivity. It eliminates whole classes of errors at run-time, by introducing compile-time type checking to JavaScript. It even lets us enforce immutable data structures and functional style - via TSLint. This is essential, and we wouldn’t start a new web front end or mobile project without it.

A couple years ago we made a bet on React and Redux, as our frameworks of choice for front end development, and this has really paid off. This pair is currently the most popular choice in the community, for good reason. This has yielded many libraries and frameworks we get to re-use and avoid reinventing the wheel.

React Native has been an excellent path to native mobile app development - front end web development skills directly translate, and we’ve now shipped multiple apps on iOS and Android. Sharing the same business logic code as your web app is a huge benefit, that most alternatives just don’t have. In fact we have developed an in-house framework for doing just that, factoring out common code into a separate package that gets consumed by both web and mobile versions of the same app.

Speaking of bets, another was Golang, which has also turned out quite well! Go’s popularity keeps surging, it’s performance keeps improving, and the great tooling lets us easily collaborate and ship high performance yet lightweight backend services.

As we continue to build on these same stacks with each new project, we refine our approaches and solve common problems. Building many small to mid-size client projects allows us to step back each time, re evaluate our choices and tweak things again when there is little to no code, which you just can’t do as easily in longer running legacy code bases. We’re slowly extracting these common patterns and solutions into frameworks which keep speeding up our productivity.

Continuous deployment and delivery is a huge force multiplier, and we keep refining our approach. This is generally the first step when we start a new project. It’s much easier to get working by first continuously deploying the initial “hello world” of the project and iterating the process as the project grows, instead of trying to bolt it on later. It’s super gratifying for us and for clients to see software come to life as we build it, not to mention the faster feedback loop that allows us to course correct sooner rather than later.

We experimented with Clojure, GraphQL, and a couple machine learning techniques this year, and I imagine we will probably start using one or more of these in a production project next year.

Overall, we’ve come a long way. Goodbye 2017! Happy new year!


2017 Year in Review


Over the past couple of months I've started to become a little bit more serious in applying functional programming techniques to my favorite client side language, JavaScript. In particular I've discovered the joy of using [Map](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/Map) and [Reduce](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/Reduce).

These functions (in conjunction with other classic favorites such as `.filter`, `.sort` etc) and [ES6](https://github.com/DrkSephy/es6-cheatsheet) have made my code more expressive and easier to understand. I'd like to share some of the common map and reduce patterns I've come to use frequently in this blogpost, contrasting these techniques with the corresponding traditional imperative approach.

## What are Map / Reduce Good For

Generally speaking, I turn to the `map` function if I need to transform some `Array` of one object type into another `Array` of a different object type. Likewise, I use the `reduce` function if I find it necessary to take an array of objects and boil it down to a non-array structure (either a primitive or some JSON object). A large portion of the rest of this blogpost are examples of variations on these two themes.

## Why I Prefer Map / Reduce to For Loops

Its possible to achieve the transformations I mentioned above with traditional imperative techniques, largely involving `for` loops and mutable data structures.

However there are some advantages to using Map / Reduce over these imperative techniques:

- Less code to wade through and maintain
- Elimination of the use of mutable data structures and variables which leave room for unintended side effects
- Encourages cleaner separation of concerns

I am hoping these benefits will become evident as I explore the map and reduce patterns below.

## Summation (reduce)

The simplest application of `reduce` I can think of is summing up an array of numbers:

```javascript
const total = arrayOfInt.reduce((ac, next) => ac + next, 0)
```

Contrast this against a traditional `for` loop approach:

```javascript
let total
for (let i = 0; i < arrayOfInt.length; i++) {
  total += arrayOfInt[i]
}
```

By using `reduce` the code is less verbose, and we avoid having to declare a mutable total variable.

## Indexing an array of objects (reduce)

Another common usage of reduce is to turn an array of objects into a single object indexed by some property of
the object. In this case, we may have a simple array like this:

```json
[
  {
    "id": "aron",
    "hobby": "games"
  },
  {
    "id": "scott",
    "hobby": "ninjitsu"
  }
]
```

... and we wish to represent this array as an object that looks like this:

```json
{
  "aron": {
    "id": "aron",
    "hobby": "games"
  },
  "scott": {
    "id": "scott",
    "hobby": "ninjitsu"
  }
}
```

Here is how to do it via `.reduce`, the spread operator, and [computed properties](https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Operators/Object_initializer#Computed_property_names):

```javascript
const indexedPeople = peopleArray.reduce((ac, p) => ({ ...ac, [p.id]: p }), {})
```

This powerful little statement is doing the following:

1. For each person `p` in `peopleArray` find their `id`
2. Make a copy of the `ac` map where there is a value whose key is the `id` from (1) and value is the person
3. When iterating through the array is complete, set the latest incarnation of `ac` map to the invariant `indexedPeople` variable

Contrast the above code with this more traditional approach:

```javascript
const indexedPeople = {}
for (let i = 0; i < peopleArray.length; i++) {
  const p = peopleArray[i]
  indexedPeople[p.id] = p
}
```

The functional approach using `reduce` accomplishes the goal with a single statement, which leaves no room for accidentally mutating state. This is not the case in the traditional approach, where `i` and `indexedPeople` could be inappropriately manipulated in some way to inadvertently introduce bugs, once more code (and complexity) is introduced inside of the loop.

## Counting occurrences of items in an array (reduce)

Another useful application of the same ingredients (`reduce`, spread operator, and computed properties) is to create an object which counts the frequency of items within an array based on some key.

Given this input array:

```json
[
  {
    "age": 33,
    "hobby": "games"
  },
  {
    "age": 28,
    "hobby": "chess"
  },
  {
    "age": 21,
    "hobby": "ninjitsu"
  },
  {
    "age": 21,
    "hobby": "games"
  },
  {
    "age": 21,
    "hobby": "kick the can"
  }
]
```

... we can use reduce to produce an object which lists the frequency of each hobby:

```json
{
  "games": 2,
  "chess": 1,
  "ninjitsu": 1,
  "kick the can": 1
}
```

This can be accomplished as follows:

```javascript
const accumulatedTotals = peopleArray.reduce(
  (totals, p) => ({ ...totals, [p.hobby]: (totals[p.hobby] || 0) + 1 }),
  {},
)
```

... which achieves it in the following manner:

1. For each person `p` in `peopleArray`, and find their hobby
2. Find the current total in the `totals` map for that hobby
3. Create a new copy of the `totals` map where the count hobby's current total is incremented by 1
4. When iterating through the array is complete, set the latest incarnation of `totals` map to the invariant `accumulatedTotals` variable

Like the previous indexing example, the functional approach seems less verbose and error prone than the traditional approach:

```javascript
const accumulatedTotals = {}
for (let i = 0; i < peopleArray.length; i++) {
  const p = peopleArray[i]
  accumulatedTotals[p.hobby] = (accumulatedTotals[p.hobby] || 0) + 1
}
```

## Flattening an object (map)

Until ES7 comes out with `Object.values(...)`, a common way of converting an object's values into an array of values is to do so with `map`:

```javascript
const peopleArray = Object.keys(indexedPeople).map((k) => indexedPeople[k])
```

Here we are creating a new array of `people` objects based on the keys of the `indexedPeople` map.

Here is the verbose imperative code for contrast:

```javascript
const peopleArray = []
const keys = Object.keys(indexedPeople)
for ( let i = 0; i < keys.length; i++) {
	peopleArray.push( indexedPeople[ keys[i] ) )
}
```

## Operation chaining encourages separation of concerns

In the following example, we are trying to convert an `indexedPeople` object into an array of people sorted by their age, excluding those under 21 years old.

Here's the `map` way of doing it

```javascript
const peopleArray = Object.keys(indexedPeople)
  .map((k) => indexedPeople[k])
  .sort((a, b) => a.age - b.age)
  .filter((person) => person.age >= 21)
```

Contrasted with the traditional approach:

```javascript
const peopleArray = []
const keys = Object.keys(indexedPeople)
for ( let i = 0; i < keys.length; i++) {
	const person = indexedPeople[ keys[i]
	if (person.age >= 21 ) {
		peopleArray.push(person) )
	}
}

// sort algorithm
const sort(a) => { ... does sorting ... }
sort(peopleArray)
```

An important thing to note is that the imperative example mixes the flattening and filtering operations in the same 'stage' of the computation. In contrast, the functional paradigm using `map` has more of a 'pipeline' approach, where the flattening, sorting, and filtering operations are more clearly discerned as separate steps.

This functional "pipeline" approach makes some operations easier to compose from smaller parts, which are easier to read, test, and maintain. The imperative approach on the other hand has fewer guard rails against bloating the individual pieces, and may lead some developers to make one piece do too much (as illustrated above).

## Final thoughts

These aren't by any stretch of the imagination the only applications of map / reduce, but I hope that it is a helpful starting point for developing your own usage patterns.


JavaScript ES6 Map and Reduce Patterns


At Olio Apps, we use the technical design process to understand a problem, and to formally articulate
a solution that is possible to validate with oneself and with one's collaborators.

Whereas pull requests and code reviews are reactive, **technical design allows review and guidance to happen proactively**.

Most importantly, the technical design process allows us to think through a problem carefully before jumping straight in, avoiding potentially expensive pitfalls and dead ends.

## Applications

Technical design can be applied _in the small_, where the behaviors of discrete functions and their interactions within one or more modules are described. Technical design techniques are also applicable _in the large_ where the interactions
within multiple coordinating systems are explored, diving into details of the internals of those systems _in the small_
as necessary. In short, its always applicable!

## How it fits in the Software Development Life Cycle

Technical design is an iterative process that happens roughly after the requirements and functional design have been worked out. Its broken into roughly 3 phases:

1. High level design
2. Detailed Design
3. Review

Each of these steps are described in detail below. Upon successful completion, the regular coding and pull request peer review cycles commence, dipping back into the technical design if peer reviews reveal that addition design is required.

## Phase 1: High Level Design (HLD)

The HLD phase is where the functional design is discussed and a general solution approach is agreed upon. Typical decisions made during this phase include:

- What frameworks or languages are required, and the rationale for using them
- Build or buy decisions
- Identification of the major technical risks & research activities
- The modules that require modification or creation, and a general description of what is needed to be done
- Outlining the interactions between modules or even functions over time (using UML interaction diagrams for example)
- Division of labor between individual contributors
- Integration planning and timelines

What isn't typically discussed at this level are the exact changes to particular functions. The conversation is left intentionally less specific in order to provide room for creativity and exploration for individuals undertaking the next phase. The HLD represents the phase of technical design where the _strategy_ of the solution is proposed.

Senior engineers/architects usually seed the ideas in the HLD, encouraging junior members to dialog and refine the solution.

## Phase 2: Detailed Design

This phase is where specific changes to the code are proposed in support of the High Level Design. Furthermore, the detailed design should identify any impacts of those changes to other parts of the system that are not being directly modified.

Typical activities performed during the phase could include:

- Description of the algorithms to be created and their application in specific modules
- The creation or modification of specific functions, describing the impact within the system being changed
- Exact schema changes if a datastore is being changed (eg. sql upgrades)
- Entity relationship diagrams showing the changes to the composition of entities if required
- Identification of any performance related impacts and mitigations
- Identification of areas in the code where backwards compatibility is threatened by changes
- Description of what automated and manual tests are needed as result of the change
- Identification of changes to external facing resources such as documentation
- Identification of any weaknesses in functional design or high level design

This phase is usually performed by the folks who will actually be writing the code. The process typically involves:

1. analyzing any existing code
2. documenting any thoughts that arise during the analysis
3. transforming those thoughts into a coherent solution that resembles English much more than a specific coding language

Just enough detail should be provided so that the proposed changes are specific enough to be challengeable. However, code snippets should be avoided if possible, as it could derail the conversation. It is easy to get bogged down in, for example, the specifics of language syntax & coding conventions.

If necessary, the engineer may do some prototyping and other short-lived coding activity which is ideally not shared but instead documented only as far as to share the conclusions gleaned from the prototype.

## Phase 3: Review

Once an engineer has documented their technical design, they should regroup with the original team to discuss the degree to which the high level design is served by the detailed design.

The reviewers may find the detailed analysis to be insufficient in some way, and the detailed design could at that point be amended. On the other hand, the detailed design may also reveal weaknesses in the original high level design. In this case the high level design should be rethought, and revisions be made to the analysis (eg. go back to step 2).

## Agreement & Post Technical Design

Once the detailed design has been agreed upon by all parties involved, any actionable points in the analysis should be turned into work items (eg. trello tickets) and the regular coding - peer review cycle can commence!


Technical Design


A few gotchas with the interaction between the `FormData` and `fetch` APIs, for those used to `$.ajax` and `xhr`.

## The browser wraps any Fetch or XHR request sending FormData

In a recent project, we needed to upload images to [Rackspace Cloud Files](https://www.rackspace.com/cloud/files) from a form. After a bit of googling, we quickly arrived at code reminiscent of the following:

```javascript
export function* uploadImage({ data, params }) {
  // request a signed Rackspace Cloud Files url and object path to upload to, from our backend
  const urlResp = yield call(client.postUploadUrl, null, params)

  const { url, objectPath } = urlResp

  // wrap data to upload in a FormData object - turns out this is not needed/wanted
  const formData = new FormData(data)

  // setup necessary HTTP headers
  const headers = {
    "Access-Control-Expose-Headers": "Access-Control-Allow-Origin",
    "Access-Control-Allow-Origin": "*",
    "Content-Disposition": "attachment",
    "Content-Type": "image/png",
  }

  // upload the logo to the returned url
  yield call(fetch, url, { body: formData, method: "PUT", headers })
}
```

To our surprise, this caused a parsing error on Rackspace. Mysteriously, our image data was arriving at the server wrapped with `------WebKitFormBoundaryJ0uWMNv29fcUxC1t--`, and Rackspace didn't like that one bit.

According to StackOverflow, the conventional wisdom was to write some server-side code to parse out the `WebKitFormBoundary` nonsense, but unfortunately we don't control RackSpace and thus can't control how how our uploads get parsed.

After reimplementing the above using raw `xhr` resulted in the same behavior, we tried removing `FormData` from the equation. Lo and behold, the `WebKitFormBoundary` was gone and our images were happily slurped up by Rackspace.

TL;DR: Don't wrap binary requests in `FormData`.

## Fetch respects the headers you set, XHR knows better

In the same project, we also had a use case for uploading CSV files. This time we were not interfacing with Rackspace, but rather our own API backend using [gocsv](https://github.com/gocarina/gocsv) for CSV processing.

Nonetheless, we once again found ourselves using a combination of `fetch` and `FormData`.

```javascript

export function* importCsv({ file }) {
    const formData = new FormData()

    formData.append('upload', file)

    const headers = {
        ...
        'Content-Type': 'multipart/form-data'
    }

    yield call(fetch, url, { body: formData, method: 'POST', headers })
}
```

Again, seemingly innocuous code was somehow failing. Our API returned 400 errors, claiming it couldn't find a 'form boundary'. But how does one add a form boundary? This time using `xhr` instead of `fetch` fixed the issue -- it was adding the form boundaries for us. But why? As it turns out `xhr` was blowing away our `Content-Type` header and appending its own, containing the appropriate form boundary, while `fetch` was respecting the `Content-Type` header we set and allowing it to fail. So when using `fetch` to submit `multipart` forms, the way to properly set the `Content-Type` header is to just leave it off.

TL;DR: Don't set the `Content-Type` header and the browser will do it for you.


Reinforced Learning

2017 Year in Review

JavaScript ES6 Map and Reduce Patterns

Technical Design

FormData Fetch Gotchas

Interested in working with us?