Close
One second...

Exposing scores on the submission pages

  • by skaw
  • posted Jan 21, 2011
  • 317 Comments

As a lot of you noticed, yesterday we experimented with publicly displaying the current score a design is getting while it is still in the running. Part of the reason for that is to give designers more info about their design so they can act on it, be it finding ways to promote their design more or otherwise.


But the bigger issue here is that since about 2007, the number of people voting has been dropping pretty hard. Last year, we had significantly less votes registered when compared to 2007. Those votes are so important to our decisions in picking designs for print and we really need more people voting.


Separately, we feel like the last couple years the work we've been doing at Threadless has been mostly focused on sales, promotions and otherwise things related to the "Shop" section of the site. This year we are refocusing on the community and participate side of Threadless. There is a ton of exciting stuff happening in the community that goes unnoticed... and it's more valuable to the health of the company and the community for people to come in and start scoring designs than it is for someone to come and buy a t-shirt.


Right now when a design is submitted, it goes into this black hole where Threadless can't really talk about it... But by exposing scores publicly, it's the first step in give us the freedom to showcase the cool stuff that's happening in the community. So rather than just us talking about printed t-shirts for sale or a promotion we're running, we can talk about artists and their submissions, get people in there voting, show leader boards of currently top scoring designs, promote designs in the running to people that are just shopping on Threadless right now, etc.


That all said, right now I'm leaning towards a system where the average score becomes exposed after you've scored the design... kinda like how most poll systems work. So the score is still public in a way but it won't influence your own score. You could always go back and change your score but I don't think that will have nearly as large of an effect as exposing it up front would.


We'd love to hear any of your thoughts on how we can increase scoring numbers on designs and also how to make the cool stuff happening in the community more front and center to Threadless.

Watch this
Page:
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
briancook
briancook profile pic Alumni

Thanks for the clarification Jake - I'm sure you've read the comments in the alumni forum as well as the forums posted here - thanks for clearing the record.

EZFL
EZFL profile pic Alumni

I like what you are doing.

Disgruntled_Pacifist

I really like this idea:

That all said, right now I'm leaning towards a system where the average score becomes exposed after you've scored the design...

I think that this system would appease all parties.

briancook
briancook profile pic Alumni

I put in my vote for testing out the results AFTER you have voted...see how that goes. Otherwise having the score shown only to the designer. The thing I think you will miss about this though is the addiction/game part of the old process. Right now it seems much less fun to be involved and promote your design when you know the score already :/ Not sure why but that's how it feels.

skaw
skaw profile pic Staff

yea i think that's another benefit of making the score show after you've voted... because then it's like a game to see how your score compared to the general score...

Maltzmania
Maltzmania profile pic Alumni

i like the transparency, it will take some getting used to but it will be good in the long run I think.

NomadSlim
NomadSlim profile pic Alumni

I think threadless needs to stir the pot up a bit right now, so I love trying this out.

alexmdc
alexmdc profile pic Staff
1 design submitted -

Im not feeling the score being shown, but maybe Im just too comfortable with the way it worked before, I did like the surprise of waitting for my score to show etc, this sorta took the fun outta that

NomadSlim
NomadSlim profile pic Alumni

Another thing I wanted to note is that since the scores have been showing... I look at the current score, and then before I vote I'm putting more concious effort into "what do I think this design actually deserves" where before I may have just blown through on my gut reaction.

rodrigobhz
rodrigobhz profile pic Alumni

What alexmdc said. But is good to know that you are looking for ways to bring more people to vote. I really wish I could help you with some extraordinary idea, but I have no clue :(

Also, the idea of showing the score after you vote is cool, but people can go back and change their vote after seeing the score. It doesn't make much sense to me, or I'm losing something?

aled
aled profile pic Alumni

I posted this in the other thread but here it is again because it's so brilliant:

@skaw - if this change is here to stay - how about you show the scores to people who have voted on the design? The score becomes 'unlocked' and viewable once you have scored it yourself.

That way people will actually be incentivised to score a design if only for curiousity. This will also mean that people score on their gut reaction to the design, and not be swayed by the existing score.

soloyo
soloyo profile pic Alumni

There are a lot of contest and prizes that are related to printed designs ( air stream, 365 flickr account ) but there is nothing related to submittions.

I also feel that the score being show will decrease the amount of votes rather than increase them. Someone might notice a design with a low score and decide not to vote since it might get taken down anyway.

aled
aled profile pic Alumni

Dammit. I skimmed your post and it turns out you already had my great idea.

soloyo
soloyo profile pic Alumni

Besides the 40K club there is not other way to recognize those who spend a lot of time voting for subs. It does requires a lot of time to vote on all of the Designs.

Frank Furt

exposed or not yet or not at all, neither would affect my vote

the Sleeping Sky
the Sleeping Sky profile pic Alumni

I agree with alex..it takes the fun out of it.

skaw
skaw profile pic Staff

lol fatheed, great minds think alike ;)

yea rodrigobhz, that's true but do you really think everyone is going to change all of their scores after seeing what the average is? the point is it wouldn't influence you initially and the barrier to changing your vote is too high for it to happen significantly often. people are going to want to just vote on the next design than to bother with going back and changing their vote.

soloyo, i agree that exposing the score alone won't have much of an influence on number of voters but the things we can do by having the score exposed will have an effect.

aled
aled profile pic Alumni

People with the best intentions saying that seeing the score would not affect the way they vote underestimate the power of suggestion.

skaw
skaw profile pic Staff

yea i agree soloyo, one idea around the office has been "badges" where people earn badges for different things, number of scored designs would definitely be one of them. we were even thinking of making real iron on badges that we could ship out and collect on a t-shirt or something haha

WanderingBert
WanderingBert profile pic Alumni

I like the idea of revealing it after scoring. I might just be resistant to change, but i felt less like subbing knowing that my score was exposed to all. Not that it wouldve actually stopped me from subbing though.

Jake Friedman
2 designs submitted - Score now!

I think the best thing would be to get me printed.

5eth
5eth profile pic Alumni

How will powerscore work? Will it continue to take us from design to design without showing us the score, or will it bring up a prompt saying what we scored vs. what the average is. If that's the case, it'll definitely slow down the scoring process, which would make some people less inclined to vote.

NomadSlim
NomadSlim profile pic Alumni

I think the scoreboard should look like one of those strength tests at the circus and the cursor would turn into a little hammer when you hovered over it, and then you score would register depending on how hard you clicked your mouse. The harder you click, the better the score.

bortwein
bortwein profile pic Alumni

I have a feeling that seeing my score during the voting time frame will indeed drive me insane. I can see where you are going with the idea and I hope it works.

skaw
skaw profile pic Staff

we won't be able to make this change quickly, but i think it'd be cool if after you vote there was a little thumbnail on the next sub's page showing you the previous sub, the vote you gave it and the average score. not a completely separate screen in between but just a little notification on the next page.

WanderingBert
WanderingBert profile pic Alumni

It was already affcting how i vote. Mostly just between 1s and 2s though. Plus i just have a weak mind.

rodrigobhz
rodrigobhz profile pic Alumni

I have a feeling that after subbing a design I'll hide under the bed for 7 days and then come back to see my final score :P

jess4002
jess4002 profile pic Alumni

haha, jake, when you said badges, i thought of pokemon gym badges.

SteveOramA
SteveOramA profile pic Alumni
4 designs submitted - Score now!

The main point is great: "This year we are refocusing on the community and participate side of Threadless."

About scoring: I think that once someone scores a design that the average score be visible to them. By "unlocking" the current average score it might be a good idea to "lock" in someone's vote. That would encourage a vote once per design system that would have people really think about the score they want to give before making the decision.

Also, it could be beneficial for all voters if there was a checkbox in the edit profile area where it could say "show design scores" or something along those lines. That way the people that don't want to see the scores, favor the surprise factor that Alex mentioned above, and want to be unbiased in the way they vote can have that ability given to them or removed by choice.

Not to be greedy but maybe add a voting incentive somehow to increase the number of votes all around...

While on the edit profile section topic, it would also be cool if "Myspace info" was removed and replaced with "Facebook info" and an additional "Twitter info". That would be neat to see on people's profiles.

I'm full of ideas even if you didn't ask for them!

rodrigobhz
rodrigobhz profile pic Alumni

Great ideas, SteveOrama.

alexmdc
alexmdc profile pic Staff
1 design submitted -

also not sure if anyone said it but the score hidden to me is better because it should be about the artwork and not aout the score, the score is only secondary in my opinion when I do something a score not always reflects the actual work. showing scores and having some people vote based on scores seems like people will be more focused on the scores then what really matters? might get people more discouraged to submit that keep scoring low from the get go etc?

ArTrOcItY
ArTrOcItY profile pic Alumni

It does take the fun and suspense out of it, at least before I had 7 days to hope for a decent score and now I have none!;) My concern is all that promoting and I'll repeat what I said in my blog: What bugs me is that now I feel like I have to promote, I have to join every social network, I have to go on 100 other sites and say - here I am, vote for me! I know I won't be doing that, just the idea of that makes me cringe, it's just a way I am so I guess I lose..:/''

bcrider

My only complaint with hiding comments is that there are times they are necessary, such as in blatant design theft; I know I've voted something highly before only to find out a bit later that the design was stolen, so I've gone back to change my vote. Not that Threadless would ever print something that was stolen, but you know what I mean.

tesco
tesco profile pic Alumni

Unrelated but, where's the Reddit button?

Morkki
Morkki profile pic Alumni

I'm sure this system will change how I score, and why not? It's a different game so I'll play by the new rules. I don't know how it will change the scores, we'll have to see.

About promoting and getting new voters, I agree it's important but what tools do we have? Registering to vote is still a bit of a hassle. Would it be possible to let people vote with their Facebook logins, for example?

Frank Furt

@bcrider, there should actually be a "report design" link on every sub page

NomadSlim
NomadSlim profile pic Alumni

If you're going to hide the score until someone votes, I say do the same with comments.

mismonaut
mismonaut profile pic Alumni

By "unlocking" the current average score it might be a good idea to "lock" in someone's vote. That would encourage a vote once per design system that would have people really think about the score they want to give before making the decision.

I don't think this is a good idea.

Also, I wouldn't mind a reddit button.

bcrider

I agree, Frank Furt!

parallelish
parallelish profile pic Alumni

i like the idea of being able to see the score after you have scored it, and as far as getting more voters (i think someoe suggested this before in another blog) i think giving incentives for voting might help, for instance giving people stp's for voting 10,000 or more or less designs.

SteveOramA
SteveOramA profile pic Alumni
4 designs submitted - Score now!

authorFrank Furtdate on Jan 21, 2011time at 12:22:12 @bcrider, there should actually be a "report design" link on every sub page

that is an easy solution to the stolen design problem.

authorNomadSlimdate on Jan 21, 2011time at 12:22:24 If you're going to hide the score until someone votes, I say do the same with comments.

that would also be a good way of raising the number of votes per design.

quick-brown-fox
quick-brown-fox profile pic Alumni
1 design submitted -

I like Fatheed/Skaw Method®, that seems like a good compromise between the old way and the new way.

A while ago someone had a cool idea to have trophies/achievements like on games consoles (sorry forgot who). Silly, fun things like certain Threadless milestones. You could have a load, like an 'X' achievement for a certain type of design, say Zombie for instance. Other, score related things too, like most accurate scores for that weeks subs, miser for low scoring x amount of tees scored zero. You could earn your little badges, and perhaps for some, earn a street team point or two.

rodrigobhz
rodrigobhz profile pic Alumni

Well maybe changing the scoring system is not the best solution. I think if the website first page had more emphasis on scoring designs - not only small banners on the sidebar saying SCORE DESIGNS, but something that drawn attention - it would help encourage votes.

SteveOramA
SteveOramA profile pic Alumni
4 designs submitted - Score now!

Well maybe changing the scoring system is not the best solution. I think if the website first page had more emphasis on scoring designs - not only small banners on the sidebar saying SCORE DESIGNS, but something that drawn attention - it would help encourage votes.

agreed. i'm sure all new people type in "threadless.com" to their browser. the first page would be most important.

DontCallMeBlanket
DontCallMeBlanket profile pic Alumni
1 design submitted -

I think the score should be hidden. When it's visible it's just a number for people to obsess over, it distracts from what's import- the design. There has to be a better way to get people voting- maybe a more prominent place on the homepage? I realize that you're selling product, but right now the "Score Designs" box is such an insubstantial part of the page.

Jake Friedman
2 designs submitted - Score now!

I think the most helpful aspect of this new revealed score is that the community/scorers can see how many designs hover in the 1.8 range or lower. I'm not trying to be an ass and say that everyone's art is bad, but rather that more voters are becoming unmotivated and deterred from voting, since they have to wade through 300 weak designs to see the 50 or so strong designs. I'm uncertain that revealing the score will improve voting numbers. My reasoning is this: because a designer sees the score early, and then knows that their design is either a hit or a miss. Regardless of the result, the designer will move on and just let the design run out. This apathy from the designer will be sort of infectious to the scorer. If the designer doesn't care about their design, then why should the voter?

DontCallMeBlanket
DontCallMeBlanket profile pic Alumni
1 design submitted -

Apparently other people posting similar thoughts on promoting voting on the homepage while I was typing my post. I think this should have been the first step toward trying to encourage more voting before doing something as drastic as revealing the score.

yogololo

Voting takes a lot of time which most people don't have. I had a thought a while back to give some kind of incentive. Like a discount on merch for scoring 5,000 or 10,000 designs 20% off a signal order.

queenmob
queenmob profile pic Alumni
2 designs submitted - Score now!

Seriously, this is what makes threadless great: The fact that we are all discussing this here. I am super amazed about that, really. Thumbs up for the threadless tsaff and the community.

Anyway, I think showing the score after vating is a great idea. All in all I always get used to big changes fast and I think changes are an important part of a community - as long as they get discussed. I, for myself, enjoy not being super nervous at the end.

yogololo

signal= single

Page:
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
No account?
Join Us