User reviews trump official ratings for young travellers in search of fun
The debate over the relatives merits of consumer ratings versus accreditation-based systems for ranking tourism product was reignited last week at the New Zealand Backpacker Industry Conference.
In New Zealand, BBH runs a user-generated system ranking more than 100 backpacker properties as a reference point for customers when selecting their accommodation.
Qualmark is a government and private sector partnership between Tourism New Zealand (TNZ) and the New Zealand Automobile Association which licenses “professional and trustworthy” New Zealand tourism businesses to use the Qualmark, “tourism’s official quality mark”.
In the backpacking sector, where price, atmosphere and experience are top of a backpacker’s priority list, a consumer product rating will always hold more weight than whether the company is rated highly by other businesses or indeed the government.
Why? Because they trust their fellow travellers to tell the truth when it comes to a product they’ve experienced. If it’s good, they’ll say so, if it’s bad they’re even more likely to say so.
In the case of hostels, CEO of Qualmark New Zealand Geoff Penrose admitted Qualmark did not take into account the atmosphere or vibe of a hostel when calculating its star rating. Their system is based on business practices, sustainability, facilities etc.
So if a backpacker was to see a five-star Qualmark-rated hostel while researching accommodation, the atmosphere would not be taken into account. It could be the best run, cleanest, most modern and up-to-date hostel but be as dull as dishwater. A consumer-rated hostel will tell the backpacker if it’s fun, hosts good parties and offers activities for guests… or not.
This is why the drop in hotel rack rates won’t threaten the hostel industry dramatically - the majority of backpackers aren’t interested in luxury, they’re interested in fun.
While official ratings and accreditations send a positive message to others in the industry, they don’t carry much credence with the backpacker consumer.
There’s no denying Qualmark is a quality system for the broader tourism industry and it’s one the Australian industry could learn a lot from – I know I would consider the star ratings when choosing my accommodation - but for a sector where experience and adventure is paramount, consumer ratings will always prevail.
By Alice Terlikowski · September 22nd, 2009 at 12:05 pm
Comments
29 Sep 09
12:39 pm
Please note that you say backpackers believe that other backpackers tell the truth, Please note this is not always the case, and the owners in BBH survey have no right of reply
We have had bad ratings and comments “only 2 stoves for 60 people – we have always had 3 full stoves and 12 hobs and our backpackers only holds 38 people maximum.
We have had on one dirty spa poll and given a 1, we do not have a spa pool
We have had comments like really close the the beach and great views of the sea in the morning. We are not near the beach and do not have any sea views.
The only thing I say is a survey once a year may not give accurate results
6 Nov 09
5:49 pm
Although this thread is a little old, the topic is ever topical.
Hostel Owner seems to suggest that backpackers are dishonest in their submissions to user surveys, but this is VERY rarely so -if ever..
BBH conducts two separate but complementary rating systems.
The first system, probably unique, is an annual census style paper survey conducted on one day in late summer when usage of backpacker accommodation in NZ is at or near its peak.
Every person using BBH accommodation on survey day is invited to review and rate ALL and EVERY NZ backpacker style accommodation they have used,on a scale of one to ten. 12000 survey forms listing upwards of 550 places that offer backpacker style accommodation in NZ are distributed to all BBH member operations. Between four and five thousand completed forms are returned for collation, providing more than 40,000 reviews / ratings.
Occasionally, a backpacker will put a review or rating on the wrong line. Occasionally, memory will fail or recollection falter, resulting in a review or rating being attributed wrongly. None of this can be attributed to dishonesty and the errors are statistically very small. A very few deliberate attempts have been made to falsify or otherwise compromise the survey process, but they have always been unable to affect results without detection.
The BBH customer satisfaction rating is a relative measure. Any accommodation is subject to the same level of error as any other leaving relative ratings unaffected. In addition, ratings are derived from a minimum of 30 results so that an unusually large number of mistakes, with ratings varying a long way from the mean, are required to make a significant difference to a resulting individual rating.
Our 17 years of surveys have provided masses of data for comparison and analysis. As a result, the annual BBH survey results are very reliable indeed.
We underestimate our customers and short change ourselves if we accept the notion that backpacker reviews are based disproportionately on a “fun” element. For example, complaints about unregulated noise from late night “fun” sessions are much more common than raves about how much “fun” was had. In fact I can’t recall a single such rave. Hundreds of thousands of reviews show that backpackers are at least as interested in security, cleanliness, comfort, facilities and hospitality as Qualmark claims to be. And backpackers are FAR better equipped to make an accurate assessment than Qualmark can ever be. A Qualmark inspector might easily count the number of showers available to ensure that they are sufficient for number of beds supplied. What’s much more difficult is to know whether the supply of hot water is sufficient to supply all of the showers when all the beds are full. And speaking of beds, how many beds does a Qualmark inspector actually sleep in any specific hostel?
Alice could confidently ignore Qualmark’s stars in favour of a BBH % rating and rest assured that BBH reviews of accommodation are user assessments of accommodation standards and hospitality rather than adventure, excitement or fun.
The annual BBH survey is conducted primarily to determine a reliable relative % rating (BPP%) for publication in the BBH accommodation guide in print and on-line. Comments are provided to operators for their information, but are not published. There is, therefore, no question of a “right of reply” for Hostel Owner.
The second BBH rating system is the much copied and more common on-line rating.
BBH’s big advantage in this genre is that ALL of the ratings/reviews on the BBH rating site relate to New Zealand accommodation. Thus users, whether reading OR rating, are comparing only apples with apples rather than Calcutta with Canberra or Christchurch. Reviews are not posted until the reviewer has experience of at least 3 NZ accommodations.
The average user contributed 9.3 reviews of NZ hostels in the past 12 months and provided an average 20.6 reviews per hostel. Not quite the 30 that we require to produce a reliable %rating from our paper survey, but enough to be reasonably confident that the reviewers have enough experience to know what they’re talking about.
In the on-line survey, reviews and the calculated ratings are published, but not the individual ratings out of 10. In this survey, Hostel Owner DOES have a space for response.
Eric Foley
Director BBH NZ
8 Nov 09
9:57 pm
I would concur with last comment, the best judge of quality, and most important marketing resource are the users or clients, not the service providers.
Ideally would be a mixture of both.
Similar to international education industry in Australia where, there has been avoidance of allowing clients or students to give valuable feedback about their study experiences in Australia.
Still an Australian lackof confidence in presenting itself to the world and competing internationally? How do you keep up with what clients think and what they want, attend industry conferences and events where counterparts and industry tells everyone what quality is? Or simply ask them?
Have your say