Welcome to Steamforged Games Forums

Register now to gain access to all of our features. Once registered and logged in, you will be able to contribute to this site by submitting your own content or replying to existing content. You'll be able to customize your profile, receive reputation points as a reward for submitting content, while also communicating with other members via your own private inbox, plus much more! This message will be removed once you have signed in.

MarkM

Vengence 2017 - Stats

113 posts in this topic
20 minutes ago, Wynter said:

1. Done

2. Done

3. Done

4. Done

5. In hand

6. Done twice so far

7. Done at every available opportunity

Dear sir are you in fact a member of the renowned SF team that with such confidence you say that playtest of hunters and alchs are at hand ?

Also had SF not said that they are comfy with the balance ?

At present time there was some muttering about 6 possible changes to some players with no clear indication what the fuss is all about - this is hardly level of transparency and communication civilized champs us myself come to expect :)

 

Isante likes this

Share this post


Link to post
Share on other sites
6 minutes ago, Seryjniak said:

this is hardly level of transparency and communication civilized champs us myself come to expect :)

Name another games company where the game developers spend a weekend drinking and playing games with its community..

Poetica and jmrhode1 like this

Share this post


Link to post
Share on other sites
1 minute ago, Redmaw said:

Name another games company where the game developers spend a weekend drinking and playing games with its community..

Games Workshop had a boat load of employees at Adepticon...

2 minutes ago, Redmaw said:

Name another games company where the game developers spend a weekend drinking and playing games with its community..

Privateer Press regularly attends cons as well..........

 

Share this post


Link to post
Share on other sites
9 minutes ago, Isante said:

Games Workshop had a boat load of employees at Adepticon...

Fair do's.. I've not been to an American convention but I remember the core GW staff didn't even want to talk to their community during their own events never mind going to others... 

Share this post


Link to post
Share on other sites

Was at Vengeance Q&A. In so far as any company is going to sit there and say, "Yeah, our product has issues.", it was stated fairly clearly that they were looking at certain players. Looking suggests play testing, because what else would they be doing?

Maybe they just sit around and hypothesise like the players do, but given there have been two major updates in two years, I have a little faith that they are on it.

Share this post


Link to post
Share on other sites
18 minutes ago, Redmaw said:

Name another games company where the game developers spend a weekend drinking and playing games with its community..

Jason Soles (head creative guy at privateer) played games with randoms at Smogcon last year. He and Michael PlumPlumPlummer drank whisky with a whole gang of us at stupid o'clock saturday/sunday over games. Other PP stafg have done similar. That was at Heathrow.

 

Share this post


Link to post
Share on other sites

Hey guys, I'm thinking this thread is going a tad too off-topic at the moment. This thread was initially started to discuss the Vengeance 2017 statistics, but has turned into a "who-is-the-better-company"/"is-SFG-doing-good-or-bad" debate. Please make another thread if you're going this off-topic and thereby just changing the entire topic of the thread.

On the Vengeance statistics, I'd like to agree with @HuggyTheBear, @malladin.ben and @Stephen78 on saying that this game is excellently balanced and that most of the higher ranked players can just play any guilds and perform well. Skill and insight are a huge part of these sorts of games.

Sure, Alchemists, Brewers and Hunters seem outliers, but just adding some errata without any expansive testing is bonkers. I'd rather wait a couple of months and receive changes, that actually fix or at least decrease the significance of the problem, than receiving some quick, improvised stuff that just creates other problems.

Lord Antoine, Camjhn, Wynter and 1 other like this

Share this post


Link to post
Share on other sites
1 hour ago, Redmaw said:

Name another games company where the game developers spend a weekend drinking and playing games with its community..

I've gotten stupidly drunk with the guys in charge of Corvus Belli at a couple cons...

Other than that, not much to add that hasn't been said other than thanks to those who have put together the stats!

Stephen78 likes this

Share this post


Link to post
Share on other sites

One thing to know about these types of statistics is that top players easily skew these results.

Some call it playing a ham

sandwich. As in they could just play with a ham sandwich and do just as well, until they meet players of equal level.

Also, knowing that this is an upper level event makes players make choices to play "the best team". This can work as a positive and negative influence upon the stats. 

If you have no reps in with that "best team" you will lose with them in tighter games against equally paired skill level against a player who knows the intricacies of their team.

What this does is create the initial bolus of information with which to begin the barometer. If these same results repeat, now you have a trend to look at for design corrections.

You can use this data to look at the middle of the data points. How did those mid-level players perform when compared equally? 

Did certain players in the top teams (5-2) appear at a high percentage? Same with team representation in that same data set. If it is all relatively sprinkled that balance is good.

I do agree that looking at the Hunters on paper they just don't look to have an easy path to victory as compared to the easiest teams. I however have not seen a high level player sticking their neck out to really dedicate themselves to figuring them out. They are less forgiving and that matters in longer events. At your local shop they are still very interesting to piece together.

Share this post


Link to post
Share on other sites
23 minutes ago, vscory said:

I do agree that looking at the Hunters on paper they just don't look to have an easy path to victory as compared to the easiest teams. I however have not seen a high level player sticking their neck out to really dedicate themselves to figuring them out. They are less forgiving and that matters in longer events. At your local shop they are still very interesting to piece together.

That is because most high level players realize they are subpar and dont want to waste their time...

MarkM and Napoleon like this

Share this post


Link to post
Share on other sites

Bunch of cool games out there now worth playing.  Guild Ball is definitely one of them.  Happy to leave that thread at that.

As to the actual data, it definitely gives you an idea of what the outliers are.  Acting on the data, however generally involves a little deeper analysis.  You want to edit outliers and see how it changes things.  For example, if you take the winning player and normalize their win record, how much does it affect the Alchemist stats?  

Breaking down games into match ups is also important to seeing how much change is needed.  The current data suggests that Hunters (and Brewers) need a buff and Alchemists need a nerf, but how much depends a lot on where the stats come from.  Maybe the real problem with Brewers is that they have a 0% win rate vs Alchemists but have a pretty solid win % with that data adjusted.  Everything is connected, so its important to really look at things when deciding which string to pull.

Share this post


Link to post
Share on other sites

I think people have also got to factor in the new OPD which was in effect at Vengeance and how this impacted some captain choices. In my mind thisnwould skew the results as well. 

Not knowing who is kicking/receiving when picking made that choice harder for some teams.

There was previously similar comments around how masons werent competative, marbles being garbage etc....to which end i played masons for about 2 months lost in the final against Bryce of a 32 man event, won the next 32man against another masons player in the final.

The comment of no one is really championing them is probably true but not for the reasons stated. Il be honest, i havent managed to get the new ones painted yet but i expect il run them at an event when i do. Id personally like a couple of subtle changes, like the bear generating 1inf....but overall i think they can compete as is and will happily put my money where my mouth is

HuggyTheBear, Wynter, DasLeo and 1 other like this

Share this post


Link to post
Share on other sites

My 2 cents.

I don't think anyone can say that this date PROVES anything, simply follows the trend of the current thinking that Alchemists are above the curve and the Hunters below it. Is there cause with a correlation? I think you'd need something like 5 more Vengeance size events to really have cause.

As a player who maybe plays 1-2 games a week (sometimes more) and attends maybe 1-2 events a month I think that as stated above Alchemists are slightly above the curve and there may be some small holes in the hunters matchups, but when GB is compared to other game systems that I could play for the same experience I think the curve is a lot closer then said games and I think SFG with the staff they have, have the right mindset and work ethic to change things when they do need changing.

Share this post


Link to post
Share on other sites

The stats for teams with 5 wins are interesting as brewers got two in there compared to a bunch of other supposedly better teams that only had 1 player with 5 or more wins.

Share this post


Link to post
Share on other sites
22 hours ago, EpicChris said:

Also I would argue the fact you could finish so well with no practice with a team while a testament to your skill, could also indicate the team is 'forgiving'. What I'm saying is that putting everything down to player skill is a false premise - we said the same about Morts in S2 and I think we can all agree now Obulus was busted.

Michael Coomber chose Butchers at the last minute cos he wanted to try something different and have fun - he normally plays Fish - and played James Long in the final, possibly playing on the top table for both the last rounds (I can't remember where he was round 6). Does this mean Butchers are forgiving too? Maybe they need a nerf (Fillet does).

Also, as has been said, the winner and wooden spoon 100% and 0% will skew the results somewhat. Why not look at the actual top 15? I was 3rd highest placed Alchemist at 13th (James Long 1st, Steve Easton 6th) - Engineers were 3rd, 8th, 9th, 10th, 11th & 12th. 6 Engineers in top 15 to 4 Alchemists. Look at just the top 10, with 2 Alchs and 4 Engineers - 100% more Engineers in the top 10. Even better, look at the number of Fishermen - the stats at the top of the thread say Alchemists had highest win ratio, Fish 2nd (57.1%) and Engineers 3rd (56.5%). How does that explain why there was only 1 single Fish player in the top 15, compared to 6 Engineers.

What are people trying to prove with the win ratios? That playing Alchs and Fish make it easier to win/be placed higher in events? Bollocks. Just play a team you enjoy and have fun.

Share this post


Link to post
Share on other sites
22 minutes ago, S_A_T_S said:

How does that explain why there was only 1 single Fish player in the top 15, compared to 6 Engineers.

There are a couple ways that could be explained - there are fewer Fish than Engineers, for example, or Fish had less extreme results (ie Fish more just below the cutoff while Engies either won consistently or not at all).

26 minutes ago, S_A_T_S said:

What are people trying to prove with the win ratios? That playing Alchs and Fish make it easier to win/be placed higher in events? Bollocks. Just play a team you enjoy and have fun.

It's the only data we have, and it's better to back up a claim with something. We also can't compare number in the top 15, for example, because that's going to be way more skewed by player skill and by popularity - and that's in comparison to a data point (win%) that's already highly affected by aggregate player skill within any particular Guild. I mean, if the top 15 were filled with Hunters that wouldn't necessarily mean Hunters are good; it would probably mean that many of the best players, regardless of their Guild of choice, had chosen to field Hunters.

That's not to say I agree with EpicChris - you guys are taking opposite sides that are both wrong, it looks to me. Or neither is entirely correct, anyways. Getting to top 15 off of a team could mean that the team is exceptionally forgiving, or that a player is very skilled, or even that a player was lucky or had a good matchups or was playing a team that was well-positioned for the field or dozens of other explanations. That's why that's a poor data point to determine teamwide effectiveness.

I may try to correct win rates for player skill. Just gotta figure out how to do that...

Share this post


Link to post
Share on other sites

Alternate facts/guild ratings:

1.  Brewers are the best because of beer, followed by Butchers because meat.

2.  Morticians are the worst because dying sucks.

3.  Everyone else is somewhere in the middle.

Share this post


Link to post
Share on other sites

Okay, so I tried to correct the data for player skill. I got the data from Tiebreak and then did a couple things to it.

1) I determined each player's individual win rate, which we're using here as a proxy for player skill. If Guild Ball used Skill Ratings like Chess I'd use that instead, but I'm using what I have. For example, James Long was the only undeafeated player, so his IWR was 1. I added 1 to each of the numerator and operator so the wooden spoon didn't give me trouble for dividing by 0 later on. Think of it like signing on was winning a pre-round.

2) I averaged all of the player's individual win rates. If no one had dropped this would be 0.5, but because of some drops and the correction I had to do above it was actually a bit higher.

3) I calculated the ratio between the mean win rate and each player's IWR, then I averaged all of these ratios belonging to a single Guild. This gives us a factor by which we can multiply the raw Win Rate we've already been discussing in order to remove the effect of individual player skill on the win rates.

It's important to note that now we aren't measuring Win Rates, but rather a score reflecting how much the team selection affects the outcome. 50 would mean that selecting that team had no effect on the results, it was all down to the player's skill. Higher scores mean a more outsized effect on the result, whether positive or negative.

vengeance_stats_corrected.JPG

Once again, by IQR there are no outliers, and by Standard Deviation only Alchemists qualify, though Hunters are close.

Now, this is further evidence that at least those two teams should see a look, but I'd take it with a grain of salt; I'm rusty on my stats work and I'm certain my methodology is incorrect, if workable. I think I need to sit down with some proper statistical software like IBM SPSS or something so I can rock some hypothesis tests. I wish I had this data set when I was eye-deep in that stuff for my degree.

H0: Choice of Guild does not influence win rate.
H1: Choice of Guild does influence win rate.

Share this post


Link to post
Share on other sites

OK, this is not nessesarily backed up by raw numbers but here's my thoughts on the current Engineer situation.. 

Engineers have some really good tech against the current "scoring goals is op" meta.. They also have strong options into teams like Brewers, Hunters and Masons which  have a lower than average defence stats.. This could explain why they were the second most popular team taken to the event..

Unfortunately, their tactics rely heavily on dice rolling and some of their combinations require more skill or forethought to pull off than most other guilds (probably all of them barring Hunters).. As a result, "better" players were capable of getting more out of the team, but those with less skill or less luck found themselves falling down the rankings..

Are they over powered.? No, but their popularity as a whole it could explain why there was a lot more engineers in the top 10-15, and also why Brewers and Hunters had such a hard time.. 

 

#saveAmber 

LunarSol likes this

Share this post


Link to post
Share on other sites
3 hours ago, Siberys said:

*SNIP*

Thanks for doing some additional analysis @Siberys, there is always one brave warrior out there that knows how to handle some stats. I will lend my voice in support of the general sentiment that the game could use some gardening/farming. Both in that the top and bottom teams could use a little balancing adjustments and that the farmers need to get released. 

I fully believe that player skill is a very large determinate in whether or not you win a game (as evidenced by @Stephen78), but faction imbalance still exists. The general feeling of the average player has been consistently backed by the only stats available in BO showings and the largest single tournament played. If we are to ignore both player sentiment and evidence and instead listen only true the truly gifted that insist that balance is fine, then that speaks to design philosophy. You can either balance a game for the average player or for the top tier of competive play as it tends to be hard to do both. I can't speak to which is right or SFs intention, but that would be interesting to know which of those philosophies they prefer. 

So all that said, I don't think steamforge has been very transparent about the models they are looking at and I personally don't know if 6+ months of a game imbalance is quite quick enough turn around. I trust they'll handle it correctly as they've done a good job for 90% of the models and just need some tweaking to bring everything towards the center. Hopefully they were just holding out to have this large data point to decide if changes need to be implemented and hopefully they can correctly interpret it. 

TLDR: data + player sentiment should = change and hopefully that'll be soon. 

JS and Trygle like this

Share this post


Link to post
Share on other sites
31 minutes ago, Killermonkey said:

 

So all that said, I don't think steamforge has been very transparent about the models they are looking at and I personally don't know if 6+ months of a game imbalance is quite quick enough turn around. I trust they'll handle it correctly as they've done a good job for 90% of the models and just need some tweaking to bring everything towards the center. Hopefully they were just holding out to have this large data point to decide if changes need to be implemented and hopefully they can correctly interpret it. 

TLDR: data + player sentiment should = change and hopefully that'll be soon. 

The imbalance isn't that big and a meta that is in constant flux because of constant re-balancing is a big no-no in my books. And tweaking doesn't seem that easy to me: for instance reigning in Harry will have huge impact on a lot of teams. Likewise Vitriol is a big reason for the success of the Alchemists. If they nerf her, what then?And the Farmers haven't entered the fray yet and they seem to have a lot of tools to handle Fish and Alchemists.  A re-balancing somewhere in the future (after Steamcon?) would be good, but I'm not sure that we are there yet. 

 

Share this post


Link to post
Share on other sites

Rebalancing can have interesting effects as well beyond the direct ones. Nerfing a team can have interesting side effects where teams they previously were an oppressive match up for will suddenly shoot up in win rate which might be hard for other teams to deal with.

In general i'd say there is an alchemist problem (being too good) and there is a hunter problem (too weak). I wouldn't touch any other team than making small adjustments to these ones and then seeing where the chips fall 

LunarSol and Siberys like this

Share this post


Link to post
Share on other sites
18 hours ago, Isante said:

That is because most high level players realize they are subpar and dont want to waste their time...

I agree that they are not a great choice for tournament play. 

They take A LOT of work to get the same results that other teams make easier and with more efficient activations.

When you have to worry about a clock and only ever playing high end meta teams Hunters are not an optimal choice yet.

I do think they are fine at a local level, and if someone was to really put the work in, they can create fine results. That is clear to be fine, not amazing.

At the high end you can't have as much potential for incorrect decision trees. Hunters require you to play almost error free on top of at a high level processing point. 

I think that is more the issue than most anything else. They are in essence a mix of Brewers and Morticians, where order of activation and tricks have to align well.

Isante likes this

Share this post


Link to post
Share on other sites
15 hours ago, Bertmac said:

The stats for teams with 5 wins are interesting as brewers got two in there compared to a bunch of other supposedly better teams that only had 1 player with 5 or more wins.

This is one of the stats I want to adress, as it is even more interesting when you consider that both of these brewers players are from Germany and score good to average tournament results there.

So if you subtract these 2 from the Win Total you get 22 Wins from 9 UK Players which is an average of 2,44 while the German players go 5,0 in average. While I have to say that 2 guys are not really representative, I am not sure if it still implicates the Meta is simply not as strong as it could be for this guild.

Share this post


Link to post
Share on other sites

The stats for those with 5 wins (down to 26th place) are as follows:

Alchemists - 7
Brewers - 2
Butchers - 1
Engineers - 7
Fisherman - 3
Hunters - 1
Masons - 1
Morticians - 1
The Union - 3

I don't think there's much to be gleaned from that list other than Alchemists were considered meta and a lot of people decided to bring Engineers (whether for meta reasons like being tech vs Footballing teams, or just because people wanted to).

EDIT: Specifically regarding comparing Brewers to teams that got one win at 5+; Butchers and Masons got their single wins at higher places than either Brewer player, while Morts and Hunters got theirs at lower places. Nobody was arguing that Hunters are better than Brewers, and Morticians have, according to the Vengeance data, probably the most "balanced" showing out of any of the Guilds, so I don't think the premise would really hold.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now