The Amazon Echo API

I love my Amazon Echo! But what else can I do with it?

First of all, I’m not a web developer, so go easy on me, but also help me out by being constructive. I’m an Automation Engineer, that is, I design electrical and control systems for manufacturing machines and then program the PLCs, and I write C# interfaces to the machines. I also did some embedded work and some website programming a long time ago, but anything after 2000 is new to me. But I guess that just makes it more exciting!

echo app

When I use the web-based control app at echo.amazon.com I can use Google Chrome to watch the network activity. Here’s a list of URLs that my browser hits with responses, with identifying stuff filtered out with “xxx”. I have a cookie set with a session ID, which is probably being used to get my user-specific information.

https://pitangui.amazon.com/api/check-client-version?version=v1.2-174-gcadbe83

(no response)

https://pitangui.amazon.com/api/devices/device

{"devices":[{"accountName":"xxx's Echo","deviceAccountId":"xxx","deviceOwnerCustomerId":"xxx","deviceType":"AB72C64C86AW2","online":true,"postalCode":null,"registrationId":null,"serialNumber":"xxx","softwareVersion":"2100"}]}

https://pitangui.amazon.com/api/todos?type=TASK&size=1

{"values":[{"complete":false,"createdDate":1418360739053,"customerId":"xxx","deleted":false,"itemId":"A94B0T49AALEW#3de6b833-3f38-3290-b98d-8f6c6d582246","lastLocalUpdatedDate":null,"lastUpdatedDate":1418360739053,"nbestItems":["roast beef","beef","roast groceries","groceries","brush beef"],"originalAudioId":"AB72C64C86AW2:1.0/2014/12/12/05/B0F00712447601DB/05:34::TNIH_2V.f34eec6a-82b3-4841-b68d-f3336830b454ZXV/1","text":"roast beef","type":"TASK","version":1}]}

[I was trying to say “brush teeth” :)]

https://pitangui.amazon.com/api/cards?limit=3

{"cards":[{"cardType":"StandardCard","creationTimestamp":1418424998722,"deleteCardAction":{"actionType":"DeleteCardAction","cardId":"A94B0T49AALEW#1418424998722","mainText":"Remove card","subText":"Learn more","subTextRoute":"help/node/201602230"},"descriptiveText":null,"giveFeedbackAction":{"actionType":"NavigateAction","mainText":"Thank you! Your feedback helps Echo understand you better.","musicCustomerId":null,"route":"beta-feedback","serviceName":null,"subText":"Send more detailed feedback.","subTextRoute":null},"id":"A94B0T49AALEW#1418424998722","imageAction":{"actionType":"PlayItAgainAction","mainText":"Radio Disney","queueCustomerId":"A94B0T49AALEW","queueId":"1f1ce0db-a300-4d1c-abf4-9b019caf6882","subText":null,"subTextRoute":null},"imageCaption":null,"imageReference":{"deviceSerialNumber":"B0F00712447601DB","deviceType":"AB72C64C86AW2","fallbackIcon":"TUNE_IN","referenceId":"1f1ce0db-a300-4d1c-abf4-9b019caf6882:1","referenceType":"MusicReferenceIdImageReference"},"nBestOptions":[],"playbackAudioAction":{"actionType":"PlayAudioAction","mainText":"Echo heard: \"i heart radio\"","subText":null,"subTextRoute":null,"url":"/api/utterance/audio/data?id=AB72C64C86AW2:1.0/2014/12/12/22/B0F00712447601DB/56:35::TNIH_2V.44b8e078-8eda-4c7f-ac8a-cc94684347deZXV/1"},"primaryActions":[{"actionType":"NavigateAction","mainText":"Browse your TuneIn stations","musicCustomerId":null,"route":"music/TUNE_IN","serviceName":null,"subText":null,"subTextRoute":null}],"registeredCustomerId":"A94B0T49AALEW","secondaryActions":null,"sourceDevice":{"serialNumber":"B0F00712447601DB","type":"AB72C64C86AW2"},"subtitle":"Live station on TuneIn","thumbsUpDownActivityAction":{"actionType":"ThumbsUpDownActivityAction","activityId":"A94B0T49AALEW#1418424998722","mainText":"Did Echo hear you correctly?","subText":null,"subTextRoute":null},"title":"Radio Disney","wrapTitle":false}]}

https://pitangui.amazon.com/api/media/provider-capabilities

{"providerCapabilities":{"ADVERTISEMENT":{"canCreateStation":false,"canJump":false,"canNext":false,"canNextAtEndOfQueue":false,"canPlay":true,"canPrevious":false,"canRepeatAll":false,"canScanContent":false,"canShuffle":false,"canThumbsDown":false,"canThumbsUp":false,"queueModifiedInPlace":false}
...and so on

https://pitangui.amazon.com/api/bluetooth

{"bluetoothStates":[{"deviceSerialNumber":"xxx","deviceType":"AB72C64C86AW2","online":true,"sconeBattery":85,"sconePaired":true,"softwareVersion":"2100","streamingState":null}]}

https://pitangui.amazon.com/api/customer-status/

{"eulaAcceptance":true,"hasActiveDopplers":true}

https://pitangui.amazon.com/api/authentication

{"authenticated":true,"canAccessPrimeMusicContent":false,"customerEmail":"xxx","customerId":"xxx","customerName":"Owen Piette"}

https://pitangui.amazon.com/api/album-art/1f1ce0db-a300-4d1c-abf4-9b019caf6882:1?deviceSerialNumber=xxx&deviceType=AB72C64C86AW2&screenWidth=1920

https://pitangui.amazon.com/api/music-account-details

That’s it. Later I’ll see if I can match these API queries against the JavaScript so I can make a more complete list of the API. I’m starting to get the idea…

https://pitangui.amazon.com/api/notifications

{"notifications":[{"alarmTime":1418513400000,"deviceSerialNumber":"xxx","deviceType":"AB72C64C86AW2","id":"xxx-B0F00712447601DB-a1","notificationIndex":"a1","remainingTime":0,"status":"OFF","triggerTime":0,"type":"Alarm","version":"26"},{"alarmTime":0,"deviceSerialNumber":"xxx","deviceType":"AB72C64C86AW2","id":"xxx-B0F00712447601DB-t1","notificationIndex":"t1","remainingTime":0,"status":"OFF","triggerTime":0,"type":"Timer","version":"28"}]}

https://pitangui.amazon.com/api/third-party

{"services":[{"associationState":"UNASSOCIATED","serviceName":"I_HEART_RADIO","username":null},{"associationState":"UNASSOCIATED","serviceName":"TUNE_IN","username":null}]}

https://pitangui.amazon.com/api/device-preferences

{"devicePreferences":[{"autocastToThisClient":false,"deviceAccountId":"xxx","deviceSerialNumber":"xxx","deviceType":"AB72C64C86AW2","postalCode":"52201","responseStyle":"CONCISE","searchCustomerId":"xxx","temperatureScale":"FAHRENHEIT","voiceCastEnabled":true}]}

And so on!

Here’s one that I guessed: https://pitangui.amazon.com/api/wake-word

{"wakeWords":[{"deviceSerialNumber":"xxx","deviceType":"AB72C64C86AW2","wakeWord":"ALEXA"}]}

I want to modify that value! I guess tomorrow.

-Owen

Hacking Alexa, the new Amazon Echo

I heard about the new Amazon Echo and immediately submitted my request. I got an invitation a few weeks later and Alexa (the Echo’s name) arrived the day before we were having all of our friends over for a party. The Out Of Box Experience was great. I have a friend with ADHD and he went nuts with it. I don’t think we ever heard a complete song form beginning to end.

As an aside, my daughters discovered a secret easter egg: Ask Alexa “What does the fox say?” Hilarious!

To be absolutely clear form the beginning, I love my Echo and I am very excited about what Amazon will be doing with it in the future. My favorite two things, besides the music, about Alexa: “Play This American Life” and “Play The Writer’s Almanac”. This blog post is about exploring what ELSE I can do with Alexa.

So here’s what I’ve done and what I learned and what I plan to do. I’ll fill in details later, I just want to get this out there, so people can see that there’s hacking going on with Alexa.

Amazon Echo Wireshark

1. Portscan Alexa, both in Setup mode and regular mode. In Setup mode Alexa creates an Infrastructure wifi network AMAZON-DP1 that you can connect to. You get the IP address 192.168.11.44 from her DHCP, and she’s at 192.168.11.1 . She responds on ports 8080 (HTTP) and 443 (SSL). More on these later. She runs Linux on an ARM processor, an A8 I think.

2. Alexa runs Avahi to advertise over mDNS. It declares services on 9 (workstation), 22 (sftp), and 22 (ssh). None of these connect, though. :( But boy would it be fun to ssh into her!

3. Wireshark her: Among other places, she hits pins.amazon.com and pindorama.amazon.com, which have something to do with the web-based control websites. The remote control website, basically just a web-based version of the Echo app, is at echo.amazon.com . More on that later. ALL traffic is SSL encryped. I’ll be monitoring her long-term later on to see if she listens when she shouldn’t… :)

Here’s a Wireshark capture file from Alexa starting up.

4. Fake the Echo app configuration process: This was fun until I realized Amazon has a great API for all of this. More on that later. But if I connect to the Amazon-DB1 wifi network in Setup mode, HTTP POST to port 8080 (Only POST is supported, no GET!), POST /OOBE HTTP/1.1 and then some Thrift JSON you can access all of the setup configuration. [0, 1:{“str”,”ping”}, 0, {}] or something like that.

5. Open her up! She’s sealed down tight, glued together on the bottom, so I don’t really want to do this, but I will. The source code shows that USB ports are being used for development.

6. The Linux source that’s used and modified on the Echo is all posted, as it should be, at Source Code Notice . It’s neat to see what tools they have running on Alexa. But of course they don’t post any Amazon code, which is where all of the real work is happening.

7. There is an API! See http://echo.amazon.com/js/eb25f-app.min.js . Maybe it isn’t a public API, but it exists. I’m working on parsing it out (Visual Studio took like 2 minutes just to cleanup-refactor the javascript!) I hope to document the API as I go along here. Keep your eyes out for new posts. There’s a goldmine in that API. For example:

alpha: “projectdee-ui-dev-registration.aka.amazon.com”,
beta: “projectdee-ui-registration.aka.amazon.com”,
gamma: “projectdee-ui-gamma-registration.aka.amazon.com”,
“pre-prod”: “projectdee-ui-pre-prod-registration.aka.amazon.com”,
prod: “guipitan.amazon.com”

In general, it really looks like Alexa communications are sealed down tight via SSL. There’s a web API for monitoring her TO-DO list and grocery list and history and cards and etc. More on that later. But it really, truly seems that her brain is in the cloud, so I don’t think there is much available for hacking her program. My one hope is that the wake-up word “Alexa” or “Amazon” can be modified, because that HAS to be local, right? And if that can be hacked, maybe there’s some more local voice recognition that can be hacked.

We’ll see! Happy hacking! Post comments below and let me know what you’ve tried! I’ll update as I can.

Breaking Even

Today is an auspicious day: The Piette Honey factory has broken even! Starting from January 1st, 2012, I have a net positive $18. A big shout-out to John Zehms and Katie Amaya, without whom this threshold could not have been crossed.

Now, my accounting is particularly suspect because I’m depreciating all of my assets in one year. I did that because I never know what will happen in the next year- I might get American Foul Brood and have to burn my hives. Or maybe I’ll get tired of the whole thing and give the hives away to a friend. Plus I can’t think more than a year into the future, so it feels unfair to expect my hardware to retain value over the next 5 years.

I didn’t keep good records last year. I kept good track of my expenses, which was a total of $561. But I didn’t keep good track of the selling until I realized that I ought to start keeping track. I had receipts for my expenses, but no receipts for the sales. Plus I started out giving a lot of it away, so there wasn’t very much actual cash income! This year we organized much more and did many more cash transactions. I think charging for my honey improves the perceived value of it, and like I’m always saying to the Goods, perception is reality. Anecdotally, I always charge myself $10 a bottle. :)

Now here’s the numbers: This year I spent $557. I got about 150 pounds of honey, so my cost is $4 per bottle- if you accept my accounting methods. If you don’t, it’s about $2 per bottle. (Of course, that doesn’t include my labor costs. :) If I were to include that, I would never break even!) Earlier I implied that my income was $575 for the year, but it was really only $509 in cash. The rest was given away, and in fact, I’ll probably give away the last 50 pounds, now that I’m solvent. I don’t want to make this into a real business that actually has profits. Also, I think I spent about $100 of my income on snacks and gifts instead of putting it away in my mattress, so I actually only have about $400 in cold, hard cash. It’s very hard for me to keep cash in my pocket. What can I say, I’m trying to stimulate the economy! I also gave myself a loan for the initial investment at the beginning of the year, but I don’t plan on paying that back. :)

So what will I spend my newfound $400 (or $18) on? More beehives for next year! I hope to buy two more hives for a total of four. While I will theoretically have twice as much honey next year, I know my costs aren’t fixed. I’ll probably need to buy a honey extractor. And I’ll need to do a lot of painting of the old hives. And hopefully my mean hive will die and I’ll need to order replacement bees, which get more expensive every year.

But here’s to next year! May God bless the bees and send some rain next summer! And here’s to the next project: Maple Syrup in the spring! I did a trial run last spring and it was awesome. This year I’ll see how much I can ramp up.

Selling Honey at the Farmer’s Market

Last evening my family and I went to the Washington Farmer’s Market to sell some honey. The event starts at exactly 5:00 on Thursdays at the Washington square, and of course we were running behind (who doesn’t with three children?) But we made it in time and set up our card table. (Funny story- we don’t own a card table, but we had borrowed this one from the Jacksons for our family reunion a few weeks ago. We keep not connecting at the right times to return it, but apparently that was God’s intention. Thank you Jacksons!)

There was a very strong breeze, so I didn’t put on the tablecloth. Instead I just bunched it up under the partially-capped frame that I brought for show-and-tell. Afterwards I threw away the tablecloth- it was a cheap plastic one- because it wouldn’t have been worth cleaning off. It was sticky!

I doled out about a pound’s worth of  honey on unsalted crackers for people to sample. After a while I just liked giving people samples for the sake of seeing them enjoy it, whether or not it led to a sale. Next time I might buy smaller crackers, because the standard saltine not only used a lot of honey, but it was a little more than a mouthful and had too much taste to let the honey taste really shine through.

That’s not to say that we didn’t sell any honey! We certainly did! I think we sold 17 one-pound jars. While not enough to cover  expenses for the year, I think that if we go back a few more times we can break even. My goal is to pay off the previous year’s expenses so I can grow a little bit bigger. Some year I’ll decide to stop growing and then we can start profiting? With my luck that year I’ll have to start replacing equipment all over again!

Emily and the girls made the whole event possible. While I spent the evening extracting and bottling, Emily was hard at work at the computer designing the new labels. They’re printed out on 2″ x 3″ adhesive shipping labels with all of the important information. Then the girls, in the morning, drew their bees on them. Gwen was keeping track, during the market, how many of her bees were chosen versus Greta’s. Greta did a rainbow theme- where she colored each bee a different color, while Gwen did multiple colors for each bee. She even had a black-and-gold Hawkeye bee. Thank you Emily and the girls for all of your hard work. In return you get… honey toast for breakfast!

Towards the end of the evening the town band started playing old ragtime tunes (Jelly Roll Morton), Emily bought brownies and ice cream for the girls, and I went over and talked to the other vendors and bought some half-price bread (fantastic! “Multi-Grain Health Bread”). Overall we had a blast at the Washington Farmer’s market. If it weren’t for the Girls’ early bedtime we would go every week, selling or not. As soon as I can get another super pulled off of the mean hive and bottled, we’ll be back.

Beginning Beekeeping

I’m well into my second year of beekeeping- In fact, I’m almost done with my second year! I’ve brought in and bottled a quarter of the harvest so far, there should be at least 90 pounds more to bring in. Now I just need to sell $300 worth and I’ll be solvent enough to do it again next year! I’m not quite ready to write a second post about the adventure, but I do want to point everyone to my writeup after the first year, here. This a very flattering picture that Emily took when I found my first frame of capped honey: (At first, Emily would regularly come out with me, sans armor, with the camera to take pictures. What a brave woman!)

Dreamhost

First of all, this sounds like an ad, but for people that don’t know how to start their own websites, it really is useful information. I just wanted to mention that I really like using DreamHost for my website. I get my own domain name, as many e-mail accounts as I want, unlimited hard drive storage, and of course website hosting, and a bunch of other perks, all for $9 a month. It’s also been a launching pad for other websites that I’ve tried starting, I can host other websites with completely different domains for just the cost of the domain registration ($14 a year).

Click on this banner to start setting up your own website, or just to start checking it out. Don’t hesitate, there’s no better way to learn than to just jump in and get started:

The first thing you’ll want to do is read through all of their help documents. They’re all at http://wiki.dreamhost.com/Main_Page. Then you’ll end up installing WordPress, and you’ll start writing content! Good luck!

Welcome to the new piettes.com!

Welcome to the new piettes.com website! Please bear with me while I update things to get the same functionality that I had before. I had originally written the old website sometime in 2001, and I had been just incrementally improving it. But then I was hacked and a lot of nasty code was inserted into everything, and the scripts that I wrote to clean it up ended up trashing a whole lot of other stuff. So welcome to the new site!