- How many users are continually coming back to the app?
- How many times a day/week/month, on average, is the app opened?
- How long is the average "session"?
- How many days is the app used after it's initial use?
- How long does an app stay on the device before it's deleted?
These stats are more objective and less manipulable -- and therefore, more valuable -- than user reviews/ratings.
And I realize that the value of these will vary app to app -- i.e. the stats of a productivity app can't be compared to the stats of a casual game -- but it gives me a much better idea of how an app is being used, and therefore how I might use it vs. reviews/ratings.
This doesn't account for apps that are unusually efficient. My bus app allows shortcuts on the home screen to arrival times for a stop, so each day I spend under a minute with it open, yet it's a better app than a clumsy interface that would waste two minutes.
We'd have to be careful of apps that game the system by installing some super-small/innocuous service that would hit Google Play once/day just to keep your usage statistics falsely high -- even when you don't open the app yourself.
I don't know enough about the guts of Android to be sure you could do this, but it seems likely.
Ideally, usage would be handled by the OS, and it could actually distinguish between running in the background vs. being active. Background usage would be discarded, ideally.
Not necessarily. What if your app comes with a widget (or tile for Windows Phone apps) that tells you all the information you need. I'd have that visible at all times and would never need to open the app itself.
I have an iPhone, so I'm not sure how they're generally used.
But I would assume that if you have a widget docked, then you are likely getting some use of it, and if not, you will undock it (and maybe also delete it).
So there is still a level of usage you could measure from widgets, though maybe not quite as meaningful as that you get from regular apps.
The stats can include how many average push notifications they send over a time. If they send a lot, but you keep the app installed for a long time then you like the push notifications and that would show in the stats.
I think that's something that could certainly be accommodated by an algorithm developed to give an aggregate 'score' for apps. Certainly the fact that the app is used every single day, regardless of how long, would be a very big plus for its rating.
> Uber would do poorly by this metric because I rarely need to take a black car anywhere, so even though the app is great I rarely open it.
Would prospective purchasers realize this, though, by the nature of the app? In other words, would I also realize that I would rarely use it (on a daily basis), but see that you do continue to come back to the app and haven't deleted it over a long period of time?
> The app that shows me when my bus is coming is pretty crappy, but I use it multiple times a day.
If you use it multiple times a day, then you apparently are finding it useful enough to continue coming back to it. And in my mind, that's all I care about.
I know this isn't perfect, but again compared to user reviews, this would be way better in a large majority of the cases.
Especially in a genre where it's getting abused, like games.
> Would prospective purchasers realize this, though, by the nature of the app?
So what you're saying is, we should burden the users with a ton of metrics, then let them figure out which of those metrics are useful for each particular app, and whether you want a higher or lower score on each of those metrics on a case-by-case basis?
There are many ways usage stats can be used to provide more value than ratings. Provide a ton of metrics to those that like them and know how to interpret them, or aggregate them somehow for those that want an "X out of 10" ranking.
> Well, what do the stars actually mean? Is it a measure of whether or not the app is useful? Or some subjective measure of how "good" it is?
Right now, they're pretty meaningless, because often, the majority those that take time to review them are counterfeit. 90% of the people that use an app never take time to review it. So I never know what most people think.
> I shudder when I think of the ways you could game a system like this.
As it stands, I can't think of how there would be more gameability to a system like this vs. ratings. Ratings are so easy to game, anybody can do it, and it doesn't cost them anything.
To game something like this, I would have to "game" it over long periods of time. Not to mention, I would have to purchase a separate device, or sacrifice my own time with my device.
It's a much more painful process to game something like this.
> If longer time-in-app is a positive signal, I could easily make my app slower.
If you make your app slower, you are sacrificing user experience, which will eventually hurt your usage. How could you game your own app without it hurting the usability of it?
Unfortunately, getting a lot of those stats requires the the app logs everyone's activity. I guess this is fine for a game client that needs to connect to a server to work anyway, but for an app whose main functions don't require internet access, most people probably wouldn't want to share the above stats.
Then I'll just buy 100 $30 android devices and leave my apps open around the clock. Hell maybe I'll buy 1000 of them and rent out time on them to unscrupulous app developers.
Maybe so, but this would cost you a lot more vs. purchasing reviews, and would probably still be outweighed by the usage statistics of "real" users.
If going by usage statistics, every user -- and more importantly, real users -- opt in to "rating" the app. Now, you're depending on people actually taking time and being willing to review the app, which is mostly only done by false users.
Add geo-diversity to the algorithm; now you have to physically locate your devices across a wider area, which becomes really awkward. True, the notion of 'hire for stats' is still a problematic one.
Active installs (as in people who have the app installed on device that is being used) would cover most of these while still being relevant across different app types.
Of course some people leave their apps installed even if they don't use them, but my guess would be it's not that many (any data on this?).
> Of course some people leave their apps installed even if they don't use them, but my guess would be it's not that many (any data on this?).
True, but this should still average out in the end. That is, those that leave bad apps installed are also leaving good apps installed. But those that like to keep their devices tidy will give real meaning behind the numbers.
In other words, don't look at the numbers absolutely, but relatively.
First, how much harder is it to spoof actual devices, and take the time to spoof usage on these apps, vs. what it takes to spoof a single review?
Second, add to that the fact that spoofing 2000 app usages compared to the several thousand (or hundred thousand?) actual real users where real usage is being collected from. If you spoof 2000 reviews -- which are super easy to do, can be done in a matter of minutes with no long-term overhead -- you are aggregated with a much smaller number of real reviews actually being submitted.
It's shady no matter who does it, IMO, and I'm surprised that nobody working on that issue had a problem with it. I hope it was at least discussed on a mailing list or something.
Are you serious? Not linking to the app store is an extremely dishonest move. Like if EA could restrict that only the people who are satisfied with Battlefield 4 should be able to rate Battlefield 4 amazon after buying it; otherwise "fill an email please".
What do you think a re-view is? A review is what the user experienced; doesn't matter if the developer is going to get the bug fixed the next day or if is going to stay there for the rest of the eternity; the same thing with the user experience and expectations. If one of the users is uncommonly dumb and doesn't understand your simple user interface that's why the app store uses an average and not the opinion of one single user when it displays the app. Plus your users can always change their ratings later on if you answer (or fix) their concerns.
I think you missed my point. Some reviews are actually support requests.... requests that the developer is unable to reply to. It would be better for nearly everyone involved (and certainly the user leaving the review) if they had been sent in an email instead.
Ok, just show two buttons: "Get help" and "Rate this app"; then you can clearly identify intention without being dishonest with your users. (But that's only tangentially related to the subject at hand)
It arguably contravenes the Google Play Store policies too.
"Developers must not attempt to change the placement of any Product in the Store, or manipulate any product ratings or reviews by unauthorized means such as fraudulent installs, paid or fake reviews or ratings, or by offering incentives to rate products." [1]
Until/unless google specifies what the authorised ways of manipulating product ratings are, all this policy does is prevent three specific behaviours. Unless you define 'incentive' quite strangely, I'm pretty sure EA aren't (unfortunately) in violation.
This seems more like a disincentive to rate the product, which, as others have said, doesn't contravene the letter of the law - one would hope that Google would amend their terms to close the loophole though.
It's not like they're preventing you from rating the app however you like from the app store -- same as any other app. They just aren't actively pushing unhappy users there.
It's dishonest manipulation to ask happy users to rate you and unhappy users to email you?
People A/B test the colour of a button to try to improve conversion rates.
With the Dungeon Keeper example we have one button leading directly to the play store rating system, and another button that adds several steps to that process.
Which button do you think results in more ratings at the play store?
It is a deliberately chosen ploy to manipulate the ratings. It is scummy. Email feedback from uses can be achieved in some different way.
"We'd love it if you would rate this game [go to playstore]
And we really want to hear from you if you're having problems or have ideas for improvement[send us email]"
I guess we just disagree. If someone came up to you in person and said they loved your app, would you not perhaps ask them to rate it? And if they say it sucks and crashed, would you not ask them to provide more detail so you can try to fix it?
Asking for feedback? Oh, that’s a terribly nice way of framing it.
This is a dishonest filtering mechanism that is deliberately set up to manipulate people. To refer to this as merely asking for feedback is highly disingenuous. You are missing the point.
Asking for feedback is ok. It might be annoying for users to be disturbed by a dialog, but morally there is nothing wrong with it. I don’t think anyone was arguing that.
But this dialog is not merely asking for feedback. It does manipulative filtering.
Very good question. Quite interested in this too and I get the sense Apple would nuke the app even if they didn't specifically write a rule against this.
Apple has put EA up on stage to demo their games at many of their keynotes. I really doubt they'd ever get removed without a discussion taking-place beforehand.
I find this story quite sad because the original Dungeon Keeper predated app stores, shady rating techniques and was really awesome. (This: http://en.wikipedia.org/wiki/Dungeon_Keeper )
I know it's "cool" to hate on EA, but assuming this isn't against any Google Play terms of service, I actually think it's a great idea for any developer.
With that said, it's obviously not in the end user's best interest in terms of seeing a true rating prior to download an app. It merely helps the developer to achieve higher ratings, more users as a result, and if they actually care about the feedback, then improving their product as well.
Nice, lead with a statement designed to belittle the opposition akin to killing an argument with the term 'fanboy'. Only slightly subtler.
This is a UX antipattern. If I want to leave feedback, then I'll leave feedback. If I want to rate the app, then I'll rate the app. What this is doing is facilitating 5 star reviews and taxing 1-4 star reviews with extra clicks and potential user confusion. It's anti-consumer, as negative reviews are likely filtered out. EA's repeated anti-consumer choices are why I hate EA. I couldn't care less if it was 'cool' to do.
It's what's called a "dark pattern": purposefully designing the user interface to make it hard for users to do what they want to do, and easy to do the thing you want them to do.
It's cases like these that make me think there should be an ethics code for software developers.
My product does a similar, but not identical thing. We ask the user for feedback (not modally), and if they decide to, we ask whether they like it, neutral, or negative. This is tracked with analytics, so we have an idea what people answer. When first built, the next step was to ask everyone to send us an email with any feedback they had. After about a year and a lot of positive emails, we decided we'd ask the positive users to rate us instead. It doesn't feel black hat to me, but maybe just because we evolved to this point. I certainly do feel uncomfortable with the cynical interpretation: that we're purposefully segmenting our users to avoid bad ratings.
I think what EA is doing is different from us because they're using stars, which are linked to App Store ratings for users, because it's 5 stars or nothing, and it looks more likely that they truly are trying to avoid sending detractors to rate the app.
But if you squint a little, they are basically the same thing. And it isn't a pattern I want to see takeoff. So, I'm conflicted.
There is a number of frameworks to encourage ratings, and they range from honest requests "you've been using this app for a while, would you like to rate it now?" which Appirater does, to a number of shadier ones like the one in this post.
I think it's interesting that, besides selecting who gets to vote, they ask for the rating very early in the game.
I assume at this point you don't need to pay anything to progress correctly in the game, so they want you to rate it before you get to see the pay2win aspects.
I agree, but I think it could be executed better. Much better. And without deceiving the user.
For example ask how do you like our app (without even a hint that it's a Google Play rating) and then either ask user to rate the app or provide feedback, depending on whether they liked the app or not.
I routinely uninstall apps that do this, and give them a worst review than what they would have gotten otherwise. If they're tricking me, making it less apparent is not a plus.
Well, they trick you in a way.. if they ask you for a review or not. But how is it different from asking only when you win a level and not when you lose? I don't see anything wrong with asking for "internal" rating and feedback as long as it does not pose as official Play rating.
The difference in your example is that you're trying to catch the same user in the best mood and not filter some users out.
People keep saying "this isn't preventing anyone from rating," but that's total nonsense. If one user can get to a rating in one click while the other has to exit your app and go hunt it down in the store, you are filtering the latter out even if your filter is not 100% effective.
It’s highly immoral, irrespective of whether or not rules are in place that disallow it. Hating someone for doing something immoral is very justified.
I’m quite shocked at the warm embrace this sort of immoral behaviour seems to be getting on HN. Google has to explicitly ban such behavior, remove this app from the store and EA needs to be publicly shamed. How could anything else be acceptable?
There's so much wrong with your post I don't even know where to start.
As a consumer, I want ratings to reflect what people actually think about the product, because that's a valuable aid for my own purchasing decisions. As a developer, I _also_ want ratings to reflect what people actually think about the product, because that means I'll be competing on the actual merits of my app, rather than how good I am at deception.
What we're seeing here is good for neither consumers nor (other) developers, it's only about manipulating ratings for EA. I really can't understand how anyone would portray this as a positive thing.
Hating on EA for this sort of shenanigan isn't "cool", it's having a modicum of sense.
I think where they cross the line is not making it an option to still leave a 1-4 star review via the app. It makes perfect sense to steer those users towards giving you direct feedback, but you should still provide a path to the app store for a review regardless.
If you _are_ actually listening to feedback and iterating on the game, the new version will bury the old reviews anyway.
Timeline of my last 30 seconds:
Elation that there was a new Dungeon Keeper, one of my favorite games from the 90s.
Vague remembrance that EA bought Bullfrog.
Thoughts of it probably being terrible in the same freemium/iap way that the Simpsons is.
Validation of fears and dissapointment.
There's a similar phenomenon I noticed when dealing with employees of companies that use follow-up customer surveys to gather feedback on their employees.
When you are finished with your tech support call, or car rental, or whatever they will ask you something like 'were you completely satisfied with my service today'? Completely satisfied is then the label for the highest rating on the follow up survey. The benefit to the company is that it gives you an opportunity to express any unhappiness and get it fixed, and the benefit to the employee is that you are more likely to put down completely satisfied if you've already said you were (because of priming).
However, this case is a bit more shady because the probability of the follow up survey is being effected by how you respond.
Presumably, that only has repurcusions within that individual company - e.g. some employees get rewarded for deception. Whilst that's still bad, and the company is obviously harming themselves in the long run, it's a lot less immoral than rewarding deception across an industry, between different companies.
Filtering by rating definitely feels shady, would it be less shady if the question was "Do you like this app? Yes/No" and then the suggestion to contact the developer or the "Would you like to rate this app question?"?
I haven't yet added "a please rate this app" pop up to my app [0] but the rate of organic reviews has fallen, none in the last few months so I think I need to actively drive reviews.
If I was building that rating dialog I'd want to still give the user the immediate ability to rate the app on the second '1-4 stars' popup, so that's the only thing that makes it devious.
Otherwise, this is actually a pretty good idea, speaking as a developer who has received low ratings for things I could have easily explained to the user if they had used the 'feedback' feature. Something like this would be a good way to remind the user they can actually interact with me and I will respond, at a point where they might need it.
Was kind of surprised after reading an article complaining about how ridiculously overpriced the upgrades were, agreeing, and then seeing it at the top of the play store chart for some category. Play store has an incentive to make as much money as possible too... Oh well. Looking for quality human engagements in the games section wasn't exactly the best first decision in the tree.
I'd love to see something similar, but far less complicated and open to gaming - trusted reviewers such as friends and known industry names; the latter being akin to how reviews used to work before crowd-sourcing broke them. Imagine how much better apps and games could be if there was some form of accountabilty for quality.
Devil's advocate - most rating behavior tends to be "interrupt driven". I.e. if stuff is working correctly, the user just carries on, but if stuff is broken, the user gets frustrated and leaves a negative review (what else can they do?).
The net result is your reviews will be negatively biased...unless you do something about it like this.
I fucking love Dungeon Keeper. Why would anyone want to rate it anything but 5 stars? I have to side with EA on this one. ;) [Well, Bullfrog actually.]
Oh of course I'm joking. If the mobile app doesn't live up to the original PC game, then fuck it.
It's a similar self-selection filter as dyslexic Nigerian scams utilize, you have to be a certain type of person to go for this. To actually submit a rating, a G+ using certain type of person...
I want to see usage statistics:
- How many users are continually coming back to the app?
- How many times a day/week/month, on average, is the app opened?
- How long is the average "session"?
- How many days is the app used after it's initial use?
- How long does an app stay on the device before it's deleted?
These stats are more objective and less manipulable -- and therefore, more valuable -- than user reviews/ratings.
And I realize that the value of these will vary app to app -- i.e. the stats of a productivity app can't be compared to the stats of a casual game -- but it gives me a much better idea of how an app is being used, and therefore how I might use it vs. reviews/ratings.