Projekt Vagabond Isn’t Dead. I Swear.

I’m starting to get back to a point where I can start working on this thing again and it’s like flipping through a kindergarten yearbook. Every now and then I’ll find something that makes me think I had a stroke of genius. Other times it’s like seeing a photo of that one kid you hated more than bees. Like you’d prefer taking nails hammered into your ears than listen to that twat for a single second more than you had to.

Tonight I had more of the latter instead of the former.

This thing has taken several forms starting out as a PoC Bash Script that was around sometime shortly before Ohio Linux Fest 2014 to a full-blown 15,000+ line C++ program which would have worked but I didn’t realize how insanely asinine packaging software has to be (I don’t have that kind of time, especially these days). But then I had the bright idea to simply make a Vagrant Box (I was already using Vagrant in the backend for handling a lot of things) and just distribute that instead of all this rigmarole. Funny thing here is that there are still quite a few snags.

As I’m typing this, the box is uploading to a cloud store that I’ll make available to the public tomorrow. I ran into some problems during this process.

  • I wanted to be a somewhat normal person and upload the Box to Atlas, HashiCorps repository of Base Boxes. I thought that would have been a great way to make this available to people. But nope! I don’t know if there’s a size restriction on the Box size or what but it just wouldn’t take it. FYI – the Box is about 780MB in size. The goal here is that someone would have simply been able to issue “vagrant up gregfmartin/vagabond” and get the VM. Man that would have been nice…
  • Google pisses me off to no end these days. Tonight was no exception. It still blows my mind that I can’t update or configure an Android SDK Installation from the terminal without either (A) getting bitched at for some ridiculous reason or (B) having to press ‘yes’ to accept fifty licenses for these libraries instead of just being able to use an option that will opt-in to any license requests that would come up. The former makes it literally impossible to automate an installation. Some people on GitHub have described a work around for this but it’s really hacky and I’m a little concerned about the platform portability of solutions like those so I’m avoiding them like the flu.
  • This all lead to my idea of just making the Box and distributing the Box that I’ve manually tweaked. This has issues in and of itself in that (A) the bundled software is static unless the user wants to manually update or (B) wait for me to to update and republish an updated Box (which I REALLY don’t want to do if I’m being honest). This leads into supplementary tutorial material that will be on the project’s website.

In case anyone is wondering, the reason why the Box is so huge is because it contains a fully updated Ubuntu 14.02 64-bit base image, required prerequisite software to use the Android SDK and associate tools/IDE, the Android SDK installation with all 5.0.1 components as well as all Support Libraries that are compatible with Linux (important to note), and the recent version of IntelliJ Community Edition. So yeah, it’s a little fat. That’s the size it would be on your disk anyway.

Tomorrow I’ll get all of the stuff up on the website like documentation and how to do things with it and what you can expect by using it as well.

Something tells me I’m going to have to make some changes to this before too long. 🙂

Vagabond 1.0 – Nearly Here

Yes that’s right. After much toil, and missing my self-imposed deadline for a release, I’m nearly complete with the program and can give it a 1.0 which means that it’ll go into General Availability.

I’m not posting Vagabond News on its website because I want that strictly to be a place to get the program from and a reference for information that developers would be interested in. Just wanted to get that out of the way as well. 🙂

Right now the program is about 98% done. All that remains is to test two additional features, push to the repository (assuming they work, which they will) and then build Debian, RPM and Source Taballs for deliverables. I haven’t built Debian packages for quite some time so I’m having to go through several refreshers to bring myself back up to speed on the workflow.

After this release, I have some additional features I want to work in so development will continue alongside general maintenance coding. I want to be able to add support for choosing to use either the official Android SDK or to use the Replicant SDK. This is more of an ethical decision as I feel like Replicant is a great choice for developing in an open-source manner but might not be the best idea if you’re writing Android to be published on Play. But the option should be there. I’m also considering the idea of implementing SDK Administrative Task Shortcuts. These are commands that can be passed to Vagabond that will inflate to more robust tasks passed to the VM that maintain the SDK.

If all goes as planned, and work and life leave me alone long enough, I can potentially have this ready to go by the end of the week.

Another Vagabond Update

I wanted to take some time out from working on it to provide you with an update to the Vagabond project.

Recently I had some piqued interest on Twitter for the project and it’s lit a little more of a fire under my arse to get it going. I’d continued to work on it but I was putting in on the back-burner in favor of other projects that I’ve been working on. It’s safe to say that I’ve got enough interest now to get it done.

If you need another synopsis on it, please check out my previous post about what Vagabond is.

Changes

My PoC code was a shell script that did the whole thing for me. I’ve since migrated the project to C++ and am going to build both Debian and RedHat Packages and also provide a source tarball. After some consideration, I may actually take this and eventually migrate to a web-interface in Perl but I’m still not too sure yet. I’ve got too much work going into the C++ conversion that I need to stick with that and see where that takes me. Additionally, I’ve put the development branch up on my GitHub. The project is under the BSD 3-Clause License so you can do what you want with it so long as it carries the license and you don’t use my name on derived works without pinging me first.

Right now the project is sitting at about 3011 lines of code (tallying both insertions and deletions but the graph is laid out on the GitHub). This may seem inflated but I’m adhering to the 80-line gutter since I’m editing exclusively in vim and I hate the word wrap on there. Rightly this could be consolidated and I’d probably manage to shave off a hundred or so lines. A compilation is possible as it is now and the help feature is 100% complete. The create option is responsive to tests but doesn’t do anything useful at this point and the Vagrant Propagation command isn’t coded yet. Most of the code might seem redundant compared to the PoC but I wanted to use some of the more robust features of C++ like Exceptions and Namespacing for proper encapsulation.

I’m still looking to be able to have this done by the end of next week. At least a rev one. If anything comes up that keeps that from happening, I’ll change the date but I’m pouring nearly all of my free time into the project so I should be able to hit it.

Vagabond – Update

Aside from the getandroidsdkdl script that I was offering for people who wanted to get started with Android Development on Linux, another cool tool I wanted to throw out there was a script I call “Vagabond” which is a Vagrant deployable which automatically configures a development environment that you can use for Android Development on your Linux computer. In fact, it may prove far easier to use it instead of the getandroidsdkdl script (it actually builds off of it). The actual Vagabond script isn’t available yet but I’m letting a rather close circle of friends test it out first before I make it public which shouldn’t be too far off from now. I wanted to give an overview of what it does.

If you’re unfamiliar with Vagrant, follow the previous link to get a better understanding of what it is.

What is Vagabond?

Vagabond is little more than a bash script that you’ll download and execute. Once you execute it, all you’ll need to do is sit back and relax while it configures the VM, installs all of the requisite packages to both update the base VM and the required packages for the Android SDK, and then exposes the synced directory from the guest to the host via host’s PATH and that should take care of everything. The user can then install an IDE of their choice on the host and run the Android SDK tools from their host computer.

Got that? To further simplify, the purpose of the Vagrant image is to host and isolate the the Android SDK and to make it even more portable since it’ll exist exclusively in a VM. The Vagrant Provisioner Script, in tandem with the Vagabond Script, exposes the SDK hosted in the guest VM to the host machine. If it still doesn’t make any sense, you may need to read up on what Vagrant is so that it sinks in a little more.

Vagabond Usage

Vagabond is, and will continue to be, stupid simple to use. I’m planning on adding some additional features to it but for right now, it only requires that you provide it with a directory where you want Vagrant to install and configure the VM in. That’s it. The rest is up to the script.

vagabond.sh [vagrant-project-directory]

Vagabond Details

Vagabond relies on three different pieces of software – Vagrant, Git and Zenity. It will check for the existence of these three on the computer before attempting to do anything. If either one of these are missing from the computer, the script will fail and return an error code specific to whichever program was missing. These codes are detailed in the comments in the script.

Clearly, the reliance on Vagrant should be rather obvious. Git needs to be installed since Vagabond will pull down the Vagrantfile and bootstrap provisioning script from my GitHub. Zenity is used to display the output from Vagrant while it’s running the VM configuration and provisioning script. The choice to use Zenity was made since the output from the provisioning script couldn’t easily be piped back to the terminal that was running Vagabond. I’m still not really sure why this was the case but that’s what was happening. If I can figure it out, or if someone a bit more skilled at bash scripting than myself modifies the source, I’ll yank the Zenity dependency out of there. I can’t say I don’t entirely dislike it though.

Vagabond then attempts to make the directory that the user specified while launching the script. No real magic here. The only thing is that it will check to see if the directory exists prior to running mkdir. Some may see it as a redundant check since mkdir will cater to already existing directories but I wanted some conditionally verbose output. Sue me.

Next Vagabond will move into the new directory and use Git to clone from my GitHub repository that contains the Vagrantfile and bootstrap Provisioning Script. Details on the Vagrantfile can be found if you read the Vagrant documentation. The purpose of the bootstrap script will also become evident as well. At least what it does with regard to Vagrantfile. Cloning from the GitHub will make a vagabond directory inside the directory the user specified for the Vagrant Project and we don’t want that. Vagabond will move the cloned files out of the cloned vagabond directory into the Vagrant Project directory and then remove the cloned directory.

At this point, we’re ready to use Vagrant. Now that Vagrantfile and bootstrap are in place, a call to vagrant up will use the Vagrantfile to configure the VM. Vagrantfile will then rely on bootstrap to supply additional configurations and instructions to the VM. The Vagrant Box used is the hashicorp/precise32 (Ubuntu Precise 32-bit). All other Vagrant configurations are defaults. The bootstrap Provisioning Script does the following:

  • Update apt repositories
  • Upgrade the system as well as install base dependencies for the Android SDK
  • Use a modified version of my getandroidsdkdl script to download the Android SDK from Google. This script will also unpack the SDK into /opt in the VM and configure permissions on it.

Once this is done, Vagabond will expose the synced directory, which is linked to the directory in the VM that contains the SDK, to the host system via a modification to PATH. That’s it. After that, so long as the VM is running, the host will have access to the SDK. The user can download and use whatever tools they want. Since the SDK exists exclusively in the VM, you can port the VM to anywhere you want.

What’s Next?

As I mentioned before, I’m letting a few people test this before I give it GA. Once I get those test results back and make some modifications to the script based on those tests, the version will be locked as 1 and released for GA. Stay tuned!

Development Journal – Smsr Update 001

I’d like to think that I started working on a rough draft with Smsr and then it evolved into something else.

This “rough draft” had a bit of code in it encapsulated in an entity called ConversationThread. A function of this class in particular was really nasty since what it did was take all of the SMS messages on the device and effectively organize them in a cohesive manner that didn’t rely on querying sub-providers like Sms.Inbox or Sms.Sent. The reasoning behind this was that when I would query some of the more specific Content Providers, like Sms.Conversations, the Cursor returned from that query would only contain three fields regardless of the applied projection. In other words, despite the fact that the contract class inherited from base interfaces that gave it the additional fields you’d want, this inheritance doesn’t apply to what’s returned from queries against the provider because these columns are missing from the Cursor. So it wouldn’t contain things like ADDRESS or _ID. And because these Content Providers weren’t designed with the ability to perform SQL Joins via queries (despite the fact that the documentation would lead you to believe so), the only other alternative, really, is to perform several separate queries against several different Content Providers and join them using either MatrixCursor or CursorJoiner. The issue there is (A) how to do this without blocking on the UI thread and (B) because subsequent queries would rely specifically on information that was contained in previous queries, how then to make sure that the queries had completed before attempting to use them?

I really tried to avoid the solution where multiple queries were involved because it just seemed so… lackluster. Not to mention that the idea of tossing back and forth between five or six Cursors seemed like an explosion in the works. But there was a major issue with the way I was doing it originally – real-time updates. Because the backing data was effectively abstracted into this seemingly convoluted construct that didn’t directly bind to the backing data, if something changed in the data (i.e. received a new text message), the structure would effectively have to be completely rebuilt. One pass on building that thing was expensive enough since it relied heavily on Collections. Doing it over and over again could lead to some really janky issues.

So I went back to the drawing board. And what I came out with, after struggling for a bit to understand MatrixCursor, was a solution that does in fact query multiple Content Providers which will yield several Cursors and then combines them into a MatrixCursor which is then given to my custom CursorAdapter. The result here also adds to a modified layout for the Conversation Stubs which uses a LayerDrawable to get the contact’s image (default if none) and apply a gradient over top of it that leads to the other half of the list.

Unfortunately, this change has broken the Activity that will show the Conversation Thread because the data that’s now contained in the stubs is different than what that Activity was looking for in the first place. But the stub viewer looks nice! 😀

If you want to follow along, the branch that this work is being done on is the “rewrite” branch.

https://github.com/gregfmartin/smsdemo/tree/rewrite

Update

Kid-rows! Don’t ask me what that means exactly because I don’t know either. I just made it up.

I’m still plugging away at apps. Smsr is in the process of undergoing a major rewrite. Last weekend I was playing with the SMS Content Provider again and having found some glint of genius in a single moment or realisation, I found a significantly more efficient way to write how it works. Implementing it throughout the project will effectively result in an entire rewrite but what’s computer science without the “science?”

Even cooler than that, my talk proposal for the Ohio Linux Fest (OLF), “Android Development on Linux,” has been approved and I received my invitation yesterday morning (read date on this post). OLF takes place in Columbus, OH starting on October 24th and runs the whole weekend until October 26th at the Columbus Convention Center. There are already a lot of great talks scheduled for there and my friend Brian Wagner is also giving a talk there about DevOps as well. If you’re planning on attending the event and want to attend either of our sessions, both Brian and I are going to be giving our talks on Saturday the 25th. Room placement and times are still tentative at this point as the event planners are still going through everything. It’s shaping up to be a really awesome event so if you’re into technology or open-source and you’re in the area, make sure to stop by!

Development Journal – Smsr

The SMS client that I’ve been working on for Android is now being renamed to Smsr (even though the root directory of the project will still say sms_demo).

I’ve uploaded the source code to Github so feel free to take a look at the source, fork or do whatever the hell you want with it. I’m going to be updating the contents of the wiki and the project landing page as major changes are made to the upstream build.

http://gregfmartin.github.io/smsdemo/