Development Journal – Smsr Update 001

I’d like to think that I started working on a rough draft with Smsr and then it evolved into something else.

This “rough draft” had a bit of code in it encapsulated in an entity called ConversationThread. A function of this class in particular was really nasty since what it did was take all of the SMS messages on the device and effectively organize them in a cohesive manner that didn’t rely on querying sub-providers like Sms.Inbox or Sms.Sent. The reasoning behind this was that when I would query some of the more specific Content Providers, like Sms.Conversations, the Cursor returned from that query would only contain three fields regardless of the applied projection. In other words, despite the fact that the contract class inherited from base interfaces that gave it the additional fields you’d want, this inheritance doesn’t apply to what’s returned from queries against the provider because these columns are missing from the Cursor. So it wouldn’t contain things like ADDRESS or _ID. And because these Content Providers weren’t designed with the ability to perform SQL Joins via queries (despite the fact that the documentation would lead you to believe so), the only other alternative, really, is to perform several separate queries against several different Content Providers and join them using either MatrixCursor or CursorJoiner. The issue there is (A) how to do this without blocking on the UI thread and (B) because subsequent queries would rely specifically on information that was contained in previous queries, how then to make sure that the queries had completed before attempting to use them?

I really tried to avoid the solution where multiple queries were involved because it just seemed so… lackluster. Not to mention that the idea of tossing back and forth between five or six Cursors seemed like an explosion in the works. But there was a major issue with the way I was doing it originally – real-time updates. Because the backing data was effectively abstracted into this seemingly convoluted construct that didn’t directly bind to the backing data, if something changed in the data (i.e. received a new text message), the structure would effectively have to be completely rebuilt. One pass on building that thing was expensive enough since it relied heavily on Collections. Doing it over and over again could lead to some really janky issues.

So I went back to the drawing board. And what I came out with, after struggling for a bit to understand MatrixCursor, was a solution that does in fact query multiple Content Providers which will yield several Cursors and then combines them into a MatrixCursor which is then given to my custom CursorAdapter. The result here also adds to a modified layout for the Conversation Stubs which uses a LayerDrawable to get the contact’s image (default if none) and apply a gradient over top of it that leads to the other half of the list.

Unfortunately, this change has broken the Activity that will show the Conversation Thread because the data that’s now contained in the stubs is different than what that Activity was looking for in the first place. But the stub viewer looks nice! 😀

If you want to follow along, the branch that this work is being done on is the “rewrite” branch.

Development Journal – Smsr

The SMS client that I’ve been working on for Android is now being renamed to Smsr (even though the root directory of the project will still say sms_demo).

I’ve uploaded the source code to Github so feel free to take a look at the source, fork or do whatever the hell you want with it. I’m going to be updating the contents of the wiki and the project landing page as major changes are made to the upstream build.

Development Journal – Android SMS App

For the longest time, I’ve been wanting to write a SMS app for Android. The issue originally (and still is unfortunately) was that if you’re running a device that has a version of Android less than KitKat (4.4.x), a programmer doesn’t have access to native public-facing APIs to get access to these Content Providers. Instead you have to rely on the Reflection APIs to poke around for them and, if they’re there, use them that way. It might not seem like it at first but Reflection can be a bit heavyweight on more resource-constrained devices (which can potentially toss ANRs or even cost battery cycles) and also makes for some really nasty looking code. I’ve worked on several projects where Reflection was the order of the day (especially for things like accessing Enterprise Configurations in WifiManager) but unless I really had no other choice, I tried to stay away from the Reflection solution as much as possible. Even going to the lengths of not writing this SMS app I wanted to do for so long because there wasn’t a public API for it. Enter KitKat with these packages and now I’m going to town.

Putting aside the fact that Google likes to arbitrarily break your code with new upstream versions, the package that contains the contract classes for these Content Providers, android.providers.Telephony, is actually quite solid. Unless they go ape-shit on this package like they did with ContactsContract (which is an absolute fucking nightmare to work with), we should be okay for the time being.

I’ll save the technical discussion for another post (or even a video) but what I wanted to do here was showcase the current state of the build that I have for the SMS app. Honestly I’ve spent more time trying to figure out the nuances of the contracts and how to use them and fucking around with 9-Patch than anything else (9-Patch is cool technically but why can’t someone at Google make decent documentation?) Additionally, I wanted to provide a tentative roadmap for where this is going.

A listing of all Conversation Stubs
A listing of all Conversation Stubs

Here’s a screen capture of the home screen which will display what I refer to as the “Conversation Stubs.” No, those aren’t native constructs in Android, it’s a custom object that I’ve built for this app. The goal here was simple – after doing all the magic necessary to sort the messages to have them make sense, grab the most recent one and from it derive the contact’s display name and a 45-character snippet of the message body. If the snippet length equals 45-characters, an ellipsis is appended to it to indicate that there’s more to the message than what’s visible. What’s missing here is an indication that the message was sent from you to the intended recipient (coming in the next nightly build). By the way, I swear I didn’t plan the colour of the ActionBar to match that of my site. 😉

Pressing on one of the Conversation Stubs will bring you to another screen which will display the entire Conversation Thread.

A Conversation Thread listing
A Conversation Thread listing

As with Conversation Stub, a Conversation Thread is not a native construct to Android. Now if you’ve done a little digging into the package yourself, you might be thinking why not just use the Conversation Contract? I’ll do more on that in the technical discussion.  Although there are no clear visual cues as to who said what, messages that have been sent from you to the recipient are on the right side and messages from the recipient to you are on the left. Each message is time-stamped currently using 24-hour format (this will be customisable in a later build). The thread recipient’s display name occupies the ActionBar in the top-left. The list of messages is sorted in descending order from the top and will automatically start from the bottom when opened which will display the most recent messages. Right now, there’s not much else here other than what you see.


Current, if short, Preferences
Current, if short, Preferences

The last point of interest here is the slowly building Preferences. One of my targets here is to have a decently high level of customisation while still retaining the overall look-and-feel of the app that I want. The bottom option is the more interesting one right now. As I mentioned before, KitKat will allow a user to specify an app to be the dedicated Messaging app. Of course I want the user to be able to choose this one as the default if they wish to do so. Pressing this brings up a dialogue that will allow you to choose the default app.

Overall, the goal here is quite simple. I prefer as vanilla of an experience on my Android devices and I like the Google Play Services and default apps but I have a serious beef with Hangouts. It just feels too clunky and I wanted something a little more back-to-basics. The current build cannot actually compose or send SMS yet (coming in two builds) nor can it determine or store draft messages (coming in a build after the previously mentioned one). MMS is also not yet handled but is coming shortly. I do not want to do anything fancy like integrate with Google Accounts like Hangouts does. I want this to be a pure unadulterated SMS/MMS experience. Nothing more.

Some tentative features that I have in mind though are sticky messages, cherry-pick deletion, hidden threads, export a conversation thread to a file (I’ve actually had a lot of interest in this from some clients for legal reasons) or print a conversation thread directly to a printer (relies on Android Print Services) or email a conversation thread to an email address, bulk messaging (one-to-many) and group messaging (many-to-many).

I’ll do a technical discussion on how to use the SMS Contracts and all of the nuances that I’ve discovered with them soon. Either way, I’m looking to have a first version published by the end of the month.

The Shopper – Questions, Questions

This morning I was able to complete another critical portion of the feature set: item deletion. It now works for either single or batched deletion. In other words, you can delete either single or multiple items in one fell swoop. Of course, there’s no way to undo this but I’m not too concerned about that right now.

Something struck me though. The original version of The Shopper had a visible dynamic calculator. As the user added, deleted, and edited items, the calculator would display the cost with and without sales tax applied. It also showed the total number of items in the list. More accurately, it showed the total of items that should be in your buggy (it counted item quantities rather than strictly the number of items in the list). I thought this was a really useful feature but for some reason or another, when I was drawing the initial UI design drafts for the current version, it never made it in. It still isn’t in there.

In fact, the degree to which it’s missing is evident in code as well. There are no accommodations for calculating a running total for the items currently existing in the list. None.

This might be a little bit of a problem. Adapting the code that calculates these values from the old code base isn’t a problem. I just need to figure out where the hell to put the calculator at. Originally, it was below the list of items (as shown below). I’m thinking that maybe it should go there again? Or maybe above the list? I’m not sure.

The Shopper v1.0 showing the dynamic calculator below the list contents

So here’s another screenshot showing the current iteration in landscape mode. Any thoughts?

The Shopper Development Update – C

This morning was by far one of the most productive mornings I’ve had in quite a while. I posted on Google+ that I was definitely in workaholic mode and that certainly wasn’t a lie.

So where to begin with The Shopper…

As a developer, I have a really nasty habit of incorporating test code into production code. This particular type of injection is required sometimes when I’m testing out a portion of the API that I’m unsure of which is fine. However what isn’t fine is when that code sticks without any type of polish. It then sticks out like a sore thumb and begins to pollute the rest of the project.

I like to think of the theory of broken windows here which I first heard of reading The Pragmatic Programmer.

So I sat back and literally, on my dry erase, sketched out the workflow as it should be at this point and catered the code to match. The results were astounding. I was actually seeing a performance gain on certain methods. Importantly I didn’t feel like I wanted to rip my hair out thinking about the mess I’d made.

A very important feature was completed and that is contextually sensitive menu items in the ActionBar. Logically it doesn’t make sense for all of the icons in the ActionBar to be present all of the time. It makes even more sense to only have certain icons present when certain conditions are satisfied. The benefits here are batch operations and to prevent users from doing something that, at that time, doesn’t make any sense (also eliminating exceptional circumstances). For example, you should be able to add an item at any point in time so the add item icon is present at all times. You can’t edit an item without first selecting one so that icon is hidden until an item is selected. You should be able to delete any number of items so when at least one item is selected, the delete icon is present. However you can’t batch edit so when more than one item is selected, the edit icon disappears (this is something that I think would be nice but would be difficult to express cohesively via UI).

For the time being, all custom styling has been dropped and the theme reverted back to the default Android Holo Light theme. Styling is a little more involved than I’m willing to give time to right now so it’s going to have to wait until a later date when more important things are complete.

The last big thing that was finished was a bit more technical.

Google has openly admitted that using the findViewById API is a bit expensive. More importantly, it can become cumbersome and redundant. A while ago I devised my own fix to the problem: why not store single references to the Views that are relevant per Fragment and Activity and issue calls to findViewById just once post inflation? The references were wrapped in a static class that was unique to each Activity and Fragment and this worked for some time. However some things were painfully obvious about the design here. This was taking redundancy to an extreme. Each Activity and Fragment had their own class which had the same design and same interface just different members. Typing this thing out over and over again was tiring but more importantly it wasn’t very good practice.

A thought occurred to me this morning: why not parameterize the members of this class? Seeing as how that was the last “least common denominator” challenge yet to be tackled, overcoming that meant that a separate entity could be created that could be distributed to each Activity or Fragment that required it. More importantly, if designed properly, it could hold instance-based references to other things like MenuItems. Thus ReferencePool was born.

The details are simple really. ReferencePool contains a collection that stores a nested collection (more specifically a LinkedHashMap that stores a SparseArray). The key for the parent collection is a Class instance while the key for the nested collection is an integer ID which is expected to be the Android-generated ID found in R. The reason for this is that neither collection should be explicitly tied to any one particular descendant class (with the exception of SparseArray but it stores Objects). That being the case, it would be really easy to lose the meaning of the object in syntax if the interface were not designed properly. Using a Class as a key helps to enforce, logically, that one SparseArray is linked to a specific type while making sense syntactically (derived contextually from reading the source). The method that gets an object from the collection uses generics (actually the whole class does) to return an Object casted specifically to the Class used as the collection key.

Despite my efforts to keep the collections as type-agnostic as possible it would be a valid statement to say that to some degree they aren’t. SparseArrays store Objects thus regardless of the Class key used in LinkedHashMap, you realistically could dump anything in there so long as you remembered the integer key for the Object in SparseArray. The goal was more syntactic. Originally I left it up to the client to cast out to whatever they needed by simply returning Object from the get method. So you either had one of two client situations: declaration with casted assignment or anonymous classes with nested parentheses; neither of which I wanted in client code and didn’t really suit the intended nature of the class and its interface. What I got instead, after implementing generics for the get method, was an inline generics declaration before the method call which looks a lot nicer and does what you would expect it to do. An example is here:


So far this is working out nicely.

Quick Thought – How I Updated Content in a TabHost

Every project introduces a few new challenges. Especially when you’re trying to add features that require you to do things that you didn’t to in the last project or simply haven’t encountered before. But this is the fun part of software development.

Lest we forget, software development isn’t fun unless you’re pushing yourself to do things that you didn’t do before.

If you’ve read some of the previous posts regarding updates I’ve made to the latest version of The Shopper, you’ll notice that I’ve decided to use a TabHost to hold ListView’s that act as dynamic filters for the proper item listing on the right side of the screen. As far as I know, tabs in Android are still a bit fringe. Judging by the implementation details presented in my instance, I’d have to say that it’s not the most elegant solution for tabs in a GUI environment I’ve used (frankly setting up tabs in GTK is easier IMHO). Just to clarify, I’m not talking about declaring a View in XML, rather I’m talking more specifically about populating dynamic tab content in a procedural manner.

Just in that last sentence, therein lies the challenge. Defining your layout in XML is one thing. Having to react to user input and dynamically change content is another. While you can anticipate the methods in which the user will do certain things that will require your app to change in some way, this isn’t a solution context that can be adequately described in a layout XML file. Typically, despite the obvious violation of encapsulating data from GUI, a procedural solution is sometimes better.

So what exactly am I going on about? The behavior of The Shopper requires that the content of the lists contained in the TabHost change on a frequent basis. In fact, every time the user performs a transaction that revolves around an item (think in the context of CRUD for you DB people), that list content is almost guaranteed to change. Thus for every action, in order to ensure the user is seeing the most accurate data, the lists are updated.

The process of actually updating these lists isn’t hard and any novice Android developer can do this. Using Fragments as I have here only adds a small layer of complexity in the form of interface-based delegation through the hosting Activity. Again nothing too seriously mind bogging.

The problem comes in when you actually try to get access to a tab’s content. It’s not as straightforward as it seems. I also may be overcomplicating things but bear with me.

TabHosts contain TabWidgets. TabWidgets are the Views that hold the actual tabs. Each TabWidget hosts an arbitrary number of tabs that can be specified in either your layout XML or via code. Procedural exploration shows that TabWidgets actually contain what are called TabSpec instances. TabSpec defines an individual tab that will be inserted in the parent TabWidget via the TabHost (the API call to perform this operation belongs to the TabHost).

My first logical instinct was to assume that a TabWidget had an API that went something like this:


It’s extremely raw and definitely shallow (also totally unrealistic) but it makes logical sense and looks nice syntactically. One would think that because a tab (TabSpec more specifically) contains content that you could reference the View and use the findViewById API to update the content.

Turns out that it doesn’t really work like that. The API for TabSpec is extremely sparse. All it allows you to do is set the content and set the indicator (inflated View content and the text/View that shows on the tab). There really isn’t anything that you can do insofar as updating the content is concerned.

I decided then to go one layer up, to the TabWidget, to see if it would provide me with anything useful. Unfortunately it doesn’t; the design here really reflects well on the Android team for making sure that individual components stay separate entities which you have to admire them for. Upon first glance, some of the methods seem like they would work for the purpose: getChildTabViewAt for example. However this, and methods like it, only select the tabs themselves, not their content.

So here I was back at the TabHost after having gone down as far as I could go and working my way back up gradually with no success. Back at the TabHost, I knew a few things:

  • Tabs can be added or removed via the TabHost in a procedural manner. Adding tabs can be performed individually but removing tabs can only be in a Draconian manner (i.e. remove either all of them or none of them).
  • Tab content can be generated in a procedural manner via TabSpec. A new TabSpec can be added to a TabHost thus making it a visible tab to the user.

While completely barbaric in some sense, my solution thus became outlined in pseudo-code as such:

  1. Obtain a reference to the known existing TabHost
  2. Instantiate required variables of respective types for operation
  3. Use the TabHost to remove all tabs and their content
  4. Repopulate the data from the database into transitive objects
  5. Configure new TabSpec instances by assigning a text indicator and inflating a View that contains the ListView for the respective tabs. The ListView adapters would be assigned here using the transitive objects as data basis.
  6. Apply custom styling to the TabWidget
  7. Cleanup

In other words, what this all meant is that due to API restrictions, I am essentially removing all existing tabs from the TabHost, manually rebuilding and assigning them back to the TabHost. Like I mentioned before, this is quite barbaric and costly in some cases but for the time being, this seems to be the more mind-settling solution for me.

What made me do this? Nothing else I found on the matter seemed to satisfy me and remain simple. Occasionally I stumbled across a few people who suggested the use of ViewSwitcher and the equivalent for Activities (the name of which escapes me now). Logically these didn’t make any sense. All I wanted to do was update content for an existing ListView and not swap between an arbitrary number of them (which is what a ViewSwitcher is designed to do and is far more suitable for View animations than anything). More importantly, the ListView was self-contained meaning that changes or interactions in one tab didn’t directly or indirectly affect the content of another tab. This same logic applies to the ActivitySwitcher only backed with the argument of additional complexity of having multiple Activities instead of multiple Views.

The other argument made is why not just check to see if the tab content’s parent layout View exists and use findViewById? The content of the tabs is generated in a procedural manner to begin with. Thus at the time of initial layout inflation, they don’t technically exist. This conditional, however, is used to determine the current orientation of the app. More specifically, the existence of the View that contains the TabHost rather than a parent View for TabSpec content.

So until I find a more efficient way of doing things, this is going to be my toss into the tab content updating solution.

The Shopper Development Update – B

Unfortunately I didn’t spend much time on The Shopper today. At least not as much as I’d intended to. However with what little time I was able to afford it, some goals were achieved.

Let’s start with the matter of orientation. I’ve been a bad developer and have been working on portrait mode as an afterthought and putting more attention to landscape. Frankly I’m more attracted to the way the app looks in landscape even though the same functionality can and will be present in portrait mode via shelves. The concept is similar to the YouTube app and is promoted even on the Android developer website under the Design section.

Portrait mode hasn’t been touched on too much

Right now the shelf isn’t implemented. Actually, swipe actions aren’t even listened to yet so it’s extremely raw. This has become a bit of a priority since I’ll be playing catch up with it if I don’t do something soon.

Meanwhile, on the landscape front, I trimmed some fat from the tab identifiers. Now they’re simply condensed to one word. Looks sexier that way. The list content is now dynamically populated from the database. As intended, the population mechanic uses distinct clauses so that multiple entries are only included once in the list. Right now, there is no enforcement on case so technically speaking there can be more than one of the same store or item entries in the list. I’m still coming up with the most user-friendly method for mitigating this behavior since realistically I only want one guaranteed distinct entry in the list regardless of factors like character case.

A bug was fixed regarding the addition of items not being immediately updated in the item listing. Now, when the user adds an item, the item listing gets updated as does the dynamic filters listing. Another bug was fixed regarding the conditional used to determine the orientation of the app. For some reason or another, I was under the assumption that checking to see if a Fragment existed was enough but that didn’t seem to be the case. Instead, a check for a particular View, more specifically the TabHost that contains the dynamic filters, was used instead and works perfectly fine.

Trimmed fat from the tab names

Sometime soon, I’ll have to do a post on how I decided to dynamically update the content of the tabs contained in the TabHost. This is a rather interesting problem in Android that, from what I’ve seen, doesn’t have a very easy solution. What I’ve done seems to do the trick but I’m not sure of the future impact of performance it will have. I’ll just have to keep my eye on it.