Saturday, July 11, 2015

Summer 2015 Weeks 1 and 2

Something different today: a look at the work I've been doing. Marginally more technical than usual because this was originally written as an update for my other blog.

What I've been working on:
Setting up the receiver half of a system that is intended to get data from the field in text format and then parse it and put it online in the form of visualizations and CSV files. A lot of the system will run on a Windows XP computer, which presents difficulties because it's an old OS.

The steps that I've accomplished are:
1. Connect modem to computer (via drivers)
2. Connect modem's received texts to the computer's file system (via Gammu)
3. Collect and parse those text files and output them to a CSV file (via Logstash and Elasticsearch)
4. Upload CSV files to the organization's hosted file system (via Core FTP)

I am currently (and have been for most of this week) working on:
5. Figuring out how to visualize the data in graphs
6. Integrate the visualizations into a web page

The original intent was to use the full ELK stack and do the visualizations with Kibana. I actually got to this point, and the graphs were really quite nice. But the next question is, how to get those visualizations onto the internet in a secure way? Port forwarding, even password-protected, has some major safety gaps, and Elasticsearch's in-house security system Shield is expensive (to get a license after the 30-day trial, you need to be a Gold or Platinum subscriber). Now I'm working on learning how to use D3.js, which is free and can run on a web page.

Notes on my workflow since I'll probably find that interesting anthropological data later:

XP computer: for anything that requires actually running the system (testing new Logstash configs, etc). Usually has a couple of different Windows Explorer folders and at least two command prompts open. Chrome on special occasions (needing to look at localhost:9200 [Elasticsearch] or :5601 [Kibana]). Wordpad for editing scripts.

My computer: research. A lot of research. I brought nine books with me to Indonesia but I've probably read more documentation and forum/mailing list posts than paper pages. Usually there are multiple windows of Firefox open, each with several different tabs, with the windows segregated by topic. If I Google something, right-click and open in a new tab so that I don't have to hit backspace. Notepad++ for programming fun.

Small notebook with pencil for writing down what I do/try, idea mapping, to-do lists, etc. Pages are indexed with a Sharpie by topic (mostly, what software I'm working with--Gammu, Logstash, etc.). Pencil has not been sharpened since I got here. Reminder: fix that.

Cup of tea always necessary. Phone used as distraction an alarmingly large amount. Glasses on and off, depending on what kind of eyestrain I'm feeling at the moment.

The next couple of weeks are a holiday because of the end of Ramadhan. I'll take the opportunity to do the heavy-duty introspection that I need. Things to think about:
-myself, as usual. More specifically:
-what I'm going to do with the "revelations" about my gender identity I've had ("revelations" because if we didn't live in an overbearingly cisnormative society && I wasn't oblivious to everything, I'd have realized in elementary school that calling my gender "girl" is less than 100% accurate)
-courage/a sense of adventure and my admiration of those who have it
-what I've learned about myself from working for two weeks
-what I need to be happy and content where I am
-how I disagree with people whom I admire
-my quest to become more high-functioning (or rather, to function at all) in high-context situations

And things that are less self-centered:
-stories I've heard about traditional Indonesian beliefs
-the intersection of environmental, economic, and social good in organizations such as the one I'm interning for
-how Indonesian Islamism also informs the above
-folk tales and science
-what makes a book good (this is also self-centered in the sense that it involves taste, which is largely subjective)
-and just for fun, since I've been rereading a lot of Paul Graham essays, environmentalist startups

No comments:

Post a Comment