I spent this morning at Union Station in Toronto, Ontario, standing in a space that allowed me to see the transition space between the Subway concourse and Union Station proper. For a little over an hour and a half, I stood, taking notes on possible questions to ask or variables to measure in a possible upcoming study. The requirements for a good question or variable are that it must be visually discernable from 50 meters away, and that it be discretely answerable either with a chosen collection of qualitative descriptors, a quantity, or as a yes/no response.

Many of the questions that I wrote must assume a context, like “count the people who are exhibiting a property at a given moment”, or “measure this across time” or in correlation with another variable.

I would love to hear other ideas for questions, or what questions anyone who might read this think sound interesting. I’m currently in the process of contacting the correct people to gain permission to perform this study by placing a camera or accessing their CCTV system.

Here they are:

1. Are people moving in an intentional direction, or passively with the crowd?
2. Given particular visual information about stairway / corridors, what do people choose (e.g. avoid a full stairway, how full, etc.)
3. Flow rate of people in/out of doors, onto/off-of, on stairs, and on flat areas per unit time
4. Colour of clothing — both in aggregate, and broken down by gender, time, fullness of platform
5. Fullness of platform, broken down by time, maybe in 30 second increments?
6. Door choice for entry/exit (total counts, counts per time range)
7. Apparent ethnicity, compared to time and to cohort’s apparent ethnicity
8. People following other people or navigating on their own or noticeably separately from the person in front of them
9. Distinctive religious symbols (crosses, turbans, burqas, payots, etc.)
10. Angles — 90 degree turns vs diagonal navigation, especially in relation to intentional vs passive motion
11. Average time spent in the system per person, at various points in time
12. Time of year, also weather: temperature, humidity, cloud cover, brightness
13. Suits, Casual wear, Tie-colour, jackets/no jackets, long sleeves vs t-shirts, pants v. shorts, dresses vs. skirts,
14. wearing headphones
15. reading books, reading newspapers, reading phones
16. origin <-> destination map over time, e.g. subway to Union, Union to ground level
17. number of trips, collisions, falls, sudden changes of direction, especially in correlation with weather, following, angles, speed, and fullness
18. backpacks, purses, laptop bags, suitcases, rolling luggage, packages
19. shoe type: running shoe, dress shoe, sandals, flats, heels
20. apparent hair colour, by time and by cohort and overall
21. apparent age, by time and by cohort and overall
22. apparent gender, by time and by cohort and overall
23. ‘footedness’ — which foot do people step onto the stairs with first & last, up and down? by gender?
24. handedness — which hand to people hold doors with versus footedness
25. location, timing, and duration of stay for buskers and beggars
26. strollers, pregnant women
27. canes, crutches, walkers, wheelchairs, seeing eye dogs
28. facial expressions, by time, by gender, by following/intentional, by footedness
29. directional facing at given instants, over time
30. pidgeon count and location, pidgeon-collectors
31. gum chewing and spitting
32. where are people looking — up, down, in front of them, sideways
33. eye colour (certainly infeasible to collect)
34. # runners, per time and overall
35. glasses-wearers, sunglasses-wearers, google-glass-wearers
36. #steps at a time, by gender, by time of day
37. haircut styles, hair length
38. loiterers
39. food — are people eating or carrying it, and their speed, following vs intentional, gender, age, time of day, etc.
40. train arrival and departure times
41. skateboards and bikes
42. apparent weight of people at given times & given behaviours
43. people whistling (probably not collectable)
44. garbage bin use, litterers
45. people asking other people questions
46. people in pairs, in groups, actively discussing things, effects on themselves and people around them
47. lanyards worn
48. people talking on phones
49. people who hold the rail or do not
50. hat wearing and styles, over time, in total, by weather
51. umbrellas by weather, gender, at given times
52. security personnel, cleaning staff, construction workers, by location and time
53. smokers, by location and behaviour, gender, time, weather
54. day of the week
55. coffee, thermos, other drinks, actively drinking, carrying multiple
56. obstruction by flow — trying to get through a crowd going against you
57. “clever” people who take a long or unusual route, perhaps confidently
58. “confused” people who stop and look around helplessly or move very slowly while taking in their surroundings
59. Stride length and tempo, limps, speed-adjustments (high and low speed per person for example) in given situations, at given times, etc
60. Where do people put their hands? In pockets, at sides, behind back, on backpack, etc.
61. Tendency of groups to clump vs tendency to expand and keep distance, over time and by gender / ethnicity maybe
62. musical instruments
63. footstep style (stompy, light, jogging, half-step at a time)
64. manifestation of fears (as a pidgeon flew by, one woman suddenly cowered and crouched looking scared, then realized everything was fine, stood up and moved along)
65. hunches / postures

Anyway, that’s about all I’ve got for now. Please tell me about other possible things to watch / checkout that may be informative for the usefulness and efficiency of the space, or suggest that topics I’ve already put up are stupid or good ideas, to help decide what I actually measure.

I wrote my award-winning* thought process up in a post on the actual ludum dare crowd blog thing, and will maybe link to everything I write there from here. That may not be much. Look forward to a timelapse video of my failure, though! And maybe a half-playable game!

1. Took a long time to settle on an idea
2. Framework chosen: Phaser
3. Sleep Time!
4. First Screenshot of The Traveller
5. Now Comes the Hard Part

* not a fact

Recently, I had a reminder that overall, a lot of things just don’t get done.

It’s rarely a conspicuous lack of a thing getting done that reminds me — instead, it’s the surprised, positive reaction from people when things do get done. We simply couldn’t muster such shock and joy if most things just… got finished.

It’s easy to assume that other people get stuff done. Certainly people finish stuff at work, do taxes, and fill out medical forms. But outside that — keeping up with friends and colleagues, participating in clubs and sports, or working on side projects — it’s tough to tell. You see each of your own failures, and mostly hear about other people’s successes. So it’s easy to imagine that you’re worse than average at doing stuff.

But we all forget to send emails and to look things up. We all fail to keep in touch with friends who we meant to stay connected with. It may seem too late to send a greeting, or to finish something you said you’d get done. It may seem like you just haven’t done enough yet. At any rate, it would feel embarrassing to send it now.

Fortunately though, these rare reminders about how bad we all are can teach us a powerful lesson, which is that late, half-baked crap is more than people expect.

All you need to do is beat the world’s low expectations

The easy way out of these situations is to not do anything. Just put it down and don’t think about it. As long as it’s not work, taxes, or health stuff, the problem will usually just go away. So that’s what most of us do. Nothing. And that’s most of us expect, too. Nothing.

But nothing is only what it looks like. Most people actually do get started. We do take the first steps toward finishing stuff. Maybe just a bit late, or something makes us put it aside incomplete. Then life happens, time passes, and embarrassment keeps us silent. So it looks like nothing got done at all.

So no one expects anything of you. Which is great news!

Because you probably have a pile of late, half-baked crap lying around. Stuff you meant to do a long time ago, or that just needs a little more work. You occasionally remember it and cringe, thinking it’s too late now. False. It’s time to send it off — just apologise if you’ve gotta. Set aside the prideful embarrassment for a moment and think about this:

They aren’t expecting anything from you — or anyone

If it’s so late that you feel embarrassed about sending it, the person has probably forgotten about it, or moved on. They’re not expecting anything now, so this is unexpected gravy. You may just make their day. They won’t be angry that you’re late, or didn’t do enough, they’ll be glad that you did something. The vast majority of people can’t manage that. Your late, half-baked crap may even amaze them.

No matter what, it’s better than nothing.

Let’s start with the following: this is a post about a part of REST called HATEOAS, and why your API probably doesn’t need it. If you don’t know what that is — especially if you also feel that you do know what REST is — or, you think your API should implement HATEOAS, this post lies in a subset of {informative, angering, boring} for you.

I ran across Zapier’s When REST Gets Messy post, and among other good advice, it had a buried gem regarding HATEOAS:

I’m also not convinced that it makes sense for an internally consumed API.

…why is this a gem? And what on earth is this HATEOAS acronym we’ve seen 3 times in two paragraphs? Read on to find out.

HATEOAS (pronounce it however you like — I use both hate-ee-ohs and hate-yo-ass) is one requirement of a RESTful API. The requirement is that a RESTful API should use Hypertext As The Engine Of Application State.

Acronym city. So what on earth does that mean?

Roughly this: each response from your API should include a representation of all validly reachable states. Probably as a set of URLs describing what can be done next — kinda like a dynamic site map, but in an API.

HATEOAS is kind of obscure

Most people don’t know what HATEOAS is. Many have never even heard of it — including a surprising number of people who claim to have written RESTful APIs. This is because it’s a bit of a complex topic, and can only be addressed after getting people to grok some more basic requirements of REST. Also… there’s no standard way to implement it — but I’ll cover that nit in a bit.

Most people learn about REST by reading a tutorial or article, and unfortunately end up with the following understanding of it:

1. Use HTTP verbs —  at least GET, POST, PUT, and DELETE (poor PATCH and HEAD get no love)
2. Your database tables (e.g. “posts”, “comments”) are “Resources”, and you should expose an endpoint for each one. The HTTP verbs map well to CRUD (Create is POST, Read is GET, Update is PUT, Delete .. DELETE) actions on them.

Those are steps on the way to a RESTful web API, but it also needs some HATEOAS sauce. Whether HATEOAS is left out of articles or people just stop reading before getting to it is unknown to me, but they frequently seem to miss it.

The two points above roughly define a resource oriented API. And you know what? That is good enough for most applications.

Resource oriented is good enough

Once again, it isn’t REST, but it’s good enough for most APIs. Writers of APIs often just desire a way to organize their data and their interaction with it. A resource-oriented API provides exactly that. They’re also using HTTP verbs, so web crawlers can’t mess up their data with errant GET requests. I think this is a big win.

But… shouldn’t they do all of REST? What exactly would HATEOAS gain them? (let alone that there are other components to REST) Here is my thesis: HATEOAS is useful for solving a problem that is more complex than what most API implementers have to deal with.

As I said before, HATEOAS basically gives a list of all the links available to you. If you’re writing both an API and its client or client library, you really don’t need that. You already know what links are available. If they change, it’s you who changed them, and you who will make corresponding changes in your API-consuming clients. Most of us are not writing APIs for public consumption, and if we are, we’re writing fairly stable, versioned APIs that are typically accessed via libraries, by purpose-written integrations or apps.

Now we see why that sentence from When REST Gets Messy is a gem. An internally consumed API is a simpler problem than the one HATEOAS solves.

What problem does HATEOAS solve?

HATEOAS lets you do something truly cool: with it, you could write a client that has no knowledge of an API aside from how to read its site-map. A business with several teams that all make APIs (internal or external) might need this, because any client might have to speak to many different APIs. The business has a choice:

1. The client can know every detail of every API,
2. Every API can work the same way, or
3. Every API can be described the same way.

The last choice is obviously a very good idea; it’s like a meta-API. Everyone agrees on one way for the APIs to describe themselves, and then one client library can know how to parse that description format.

It’s now okay for the APIs to be totally unstable. Clients are not constantly needing updates, because they aren’t tied to a particular data representation or quirk. The HATEOAS representation gives enough information for the client to figure out what it needs. Google or Yahoo or Amazon are the type of place for whom this is a real problem.

It’s sort of the magical dream that one day, all APIs everywhere will be self-describing, and all clients will know how to parse the descriptions and figure everything out for themselves. Then we can stop being as concerned about API stability and versions, and writing clients or fixing API bugs would conceivably be way easier.

Sounds great. Why not just implement it and go full REST?

Unfortunately, HATEOAS has some problems. The worst of which is that you have to come up with your own way to describe your API, because REST just tells you to do it, not how to do it. To my knowledge, there’s no known good way to describe all possible APIs yet — so you’re on the hook for that.

Then the smaller problem: adding in self-descriptions takes extra work. Just writing a client that parses them can be a lot of work.

There are some attempts at making a standard, but we’ve all seen that xkcd. If one takes off, people could be writing general purpose API consumers, and that would be cool. For now though, there are a million ways to describe your API, so put yourself in the shoes of someone writing a client. You have the following choice:

1. Pay close attention to the HATEOAS implementation of the API and write a consumer to parse it and figure out what to do dynamically.
2. Pay close attention to the Resource implementation of the API and write a consumer to interact with it, and update it when needed.

The latter option is much easier up-front. So consumers rarely use HATEOAS, even if it’s present. Now, put yourself back in the shoes of an API writer: is it worth spending the time to a) come up with a description format and then b) implement it? In practice, most API writers end up in one of the following situations:

1. Spend extra time to write a beautiful, fully RESTful API and have no one really use the HATEOAS components.
2. Bolt a crappy implementation of HATEOAS on to your resource-oriented API and have no one really use it.
3. Actively choose not to provide a HATEOAS representation because you know no one would use it anyway.
4. Blissfully remain ignorant of the fact that your non-HATEOAS API is not a RESTful API.

And some rare organizations fall into this category:

5. Spend extra time to write a beautiful, fully RESTful API that you and your
customers use, because your constellation of APIs are constantly in flux.

You probably don’t need REST.

It’s a fun mental game to play, and it can be smart to include links in your API responses to ease a client’s work. Fully representing your application’s state in every response is almost always overkill — so until we have solved how to do HATEOAS everywhere, a resource-oriented API is good enough.

I don’t really use mailing lists, but I have often seen references to their utility.

This is a bit of an exercise in trialling a research methodology out. I’m interested in making a program to help me do this, but I’m going to try just writing all the stuff I think I want to save in a database down in a blogpost, and see how cumbersome it is. I already do a lot of this in my head, but I sometimes decide not to bother, because no one can see what I have only done in my head.

Topic: Mailing Lists

Questions:

1. I don’t know what program (if any) is required to use (read: participate in) them effectively, or at all. I don’t particularly like using gmail these days — I have thousands of unread messages.
2. I don’t know what a good set of mailing lists would be to subscribe to.
3. I don’t know how to find them.
4. I don’t know what etiquette prevails on them
5. I don’t know how multiple threads are kept running or how to not let them overwhelm me. (in other words: how to manage them)
6. I don’t know what I don’t know.

Links:

1. From a google search for “using mailing lists”:
   1. http://www.livinginternet.com/l/lu.htm
   2. http://manuals.kerio.com/kms/en/sect-mlistusage.html
   3. http://manuals.kerio.com/kms/en/sect-mlistusage.html
   4. http://en.wikipedia.org/wiki/Mailing_list
   5. http://programmers.stackexchange.com/questions/71148/why-do-programmers-still-use-mailing-lists
2. From a google search for “reading mailing lists”:
   1. http://softwarerecs.stackexchange.com/questions/3213/client-for-reading-several-big-mailing-lists
   2. http://stackoverflow.com/questions/351318/best-practices-to-follow-read-large-mailing-lists

Know:

• I know that mailing lists have smart people discussing interesting things on them
• I know that I could just subscribe to a given one (probably by emailing it) and start getting messages.
• I know that I would probably get a lot of messages if I subscribed to more than a few active lists.

Hmmmm. Neat. So, in flushing out the things I know and the things I think I need to figure out, an important truth has been discovered: I already know how to use mailing lists sufficiently well. I mean, goodness, I use them at work every day. For some reason, mailing lists in the wild just seemed like a different beast to me.

My questions are more about “how to be a power user of these” than “how do they even work” — but even there, I think I have a fair grasp. For example:

• I know that etiquette will be somewhat variable between lists and the kind of thing you just have to feel out.
• I know that I can find lists by going to programming language or project sites and finding their information out, or asking in an associated IRC channel if one exists.

I do not know how to manage them or what I don’t know.

Want To Know

• I want to know how to manage them effectively.
• I want to know what I don’t know, haha. I’ll keep an open mind on this.

Learning

This shift in ‘what I want to know’ suggests I should perhaps issue a new google query, ‘how to manage mailing lists’ rather than ‘how to use mailing lists’. A google search for that shows why it is a bad idea: most people who search that are looking to manage a list, not manage their interaction with several lists. After a few more trial queries, I landed on the simple “reading mailing lists”, which gave me links 2A and 2B.

2B is a little worthless and short, but 2A looks like the kind of goldmine I was looking for, and it’s also telling me something I’d wondered: a specific client, like mutt, may be my best option.

Analysis

I think I can’t actually know super practically what managing them entails until I start trying, so maybe I should just subscribe to a bunch and see how it goes. If gmail fails me, and I can’t find anything out there, I’ll can build something that solves my problems. The suggestion of trying mutt is likely where I’ll turn to first once I run into a problem, but it might be nice not to have to boot a vm to read mail (I use windows on my home laptop.)

Thanks for bearing with me through this slightly silly experiment.