The number one thing that I am actually good at when it comes to all this information is simply connecting the facts together. Seeing data and drawing conclusions from it is a skill that still needs work, but simply sorting the information does worlds in that direction.
So let’s look at the data I’ve been working on recently.
For Dnd 5e14 I’ve been working on a Psion ever since WOTC abandoned the mystic. The entire class is written up, except for one nasty little segment.
The telekinetic. An iconic type of psychic in the real world that I just have no figured out the correct numbers for yet.
So what do we have.
At 2nd level a player has access to three 1st level spells, and around 3 cantrips. That is universal across full caster classes. The warlock and the psion both restore their slots on a short rest, so that’s the numbers we are going to base on.
Giving an at-will 5th level spell, telekinesis, would under normal circumstances be bonkers absurd. Fireball, top tier 3rd level, deals 8d6 fire damage in a 20ft radius. High damage, large area of effect, but fire damage is commonly resisted. Upcasting to 5th level makes it 10d6.
Blight, however, is a 4th level spell that deals 8d8 damage.
Average of fireball (8d6 (3.5 avg / 1d6 * 8)) is 24~ damage. Average of Blight (8d8 (4.5 / 1d6 * 8) is 32~ damage. Notable that while it is necrotic, also easily resisted, it has special effects when cast on undead, which are the primary resisters of necrotic damage. It is also a single target spell.
We can conclude that Blight is an average damage 4th level spell because fireball is a high damage 3rd level spell (28 damage avg when upcast to 4th level, still 4 below Blight).
So finally, 5th level. Cone of Cold is fills the same niche as fireball at 5th level; that is, a large AoE commonly resisted damage spell. With its damage being 8d8, we now know that an 8d6 3rd level AoE is equal to a single target 4th level is equal to an AoE 5th level.
So we have our damage numbers (reduced by 25% for easily resistable damage types).
Fireball: 24, 28, 32
Blight: 32, 37
Cone of cold: 32
So now we arrive back at telekinesis. As a 5th level spell, it is extraordinarily weak in combat. It for starters has *zero* built in damage, making it entirely utility (or crowd control, but only one at a time). You would need to find clever uses for it by lifting, throwing, or pulling enemies into dangerous terrain.
The second effect is a little more interesting, though. You can launch unbound objects up to a huge size category up to 30 feet. Given the square cube law, with the average human being 150 lbs and medium size, and a large size creature is 2×2, and huge size is 3×3, we end up with:
150 * 4 = 600lbs
600lbs * 2 (square cube law) = 1200lbs for a large size creature.
1200lbs * (increase from 2×2 to 3×3 = 4 to 9, so 2.25) = 2700lbs
2700lbs * 2 (square cube law) = 5400lbs
So a huge sized object is approximately 5,400lbs (worth mentioning that the upper limit of Telekinesis in 5e 2014 was only 1000lbs).
Oh boy. Now that’s heavy.
What does all this mean?
It means that you can lift, throw, pull, push, or whatever Five Entire Thousand And Four Hundred Darn Diddly Pounds.
The next step (we’re almost there) is calculating damage of large sized objects. This one is actually quite simple, as fall damage is consistent across all creatures no matter their size, due to simple desire for simplicity (3rd edition and I believe 3.5e as well actually had larger creatures take 2x, 3x, 4x, etc fall damage to be more physics accurate).
So if a falling humans takes 1d6 fall damage per 10ft fallen (first 10ft is free), and we know from the expansion book Tasha’s Cauldron of Everything that a creature falls 60ft in the first round, and 600ft by the end of the second round, and we know that a single round is 6 seconds, then we know that the rate of acceleration is 10ft * (6 * seconds).
Meaning fall damage increases by 1d6 * ft/second*10.
Oh boy we’re so close to done.
We actually have all the numbers we need now.
5400lbs for a huge sized object.
30ft of falling is 3d6 of fall damage for a medium creature, falling at a mere 60ft*second (per round)
By the end of that 30ft of falling, a huge object is travelling FOUR HUNDRED MILES PER HOUR.
And that damage?
Doesn’t matter.
It doesn’t matter you see. Because there is a fundamental flaw in all of this math that will bring us to the final conclusion we are coming to.
The end result damage? It doesn’t exist. There are no established rules for how much damage a creature or object takes for falling onto or being fallen onto. There is instead established rules for what a character of 2nd level should be capable of doing on average.
My experience playing the game tells me that the damage should be 1d10 / 10ft fallen. 1d6 for a medium creature, 1d8 for large, 1d10 for huge. So that’s 3d10 damage for dropping a massive 5,000 pound rock on a creature. Average humans has 4*(10 years) hit points, and 3d10 averages 16 damage, so that’s well enough to kill most people, but it’s not nearly the obliteration you’d expect.
So why’d I take us through all of this made up math? It was to demonstrate the result of ANALYTICS WITHOUT INSIGHT.
KPI is a variable that simply describes “Whatever data we want to measure”. Insight is what we learn from that data that leads to actions. At some point way back in all that math, any reasonable person would have stopped and asked “Hey wait a second, is this math that actually matters for the purposes of playing the game?”. Since the answer is obviously “No, man, that is way too much to determine the damage of a spell.”, we can conclude that there was missing Insight.
In this case, the particular Insight we were missing was the following:
What is easiest?
What is consistent with the other rules of the game?
And above all:
What is fun?
If we were to continue that equation, then we would have discovered that the 2nd level Psion was able to do some colossal amount of damage, probably up to 10d10, at-will. Whenever it wanted. Compare that to the 3d6 that a 2nd level character is actually able to deal at the absolute max in a single attack ONE time.
That’s just absurd, right?
The correct answer here is 3d10, because that is a high amount for the player’s level, consistent with fall damage and size category increase across the rest of the game, and it DOESN’T make the player overpowered while still rewarding them for manufacturing a situation that allows them to drop a Big Ass ™ rock on some nasty goblin’s head.
All this talk has been missing that simple bit of insight: that all these numbers are useless when trying to figure out the relative power of an ability, because the power of an ability is exactly that: Relative. It only exists in comparison to the damage of other similar abilities throughout the game, in the specific framework of the game.
Meteor Swarm, the most damaging spell in the game, does a combined total of 40d6 damage in a roughly 200ft radius.
Vicious Mockery, the least damaging spell in the game, does a mere 1d4 damage.
This sets the upper and lower bounds of damage throughout the game, all other data MUST abide by these limits. Even if we changed that 1d4 to 300 we could figure out that 1d6 is equivalent to 400, thus Meteor Swarm does 40,000 damage. But these numbers are still relative. KPIs must remain consistent and realistic only within the boundaries of the environment they are being used for.
No one is expecting 2 million impressions on a post on Tumblr, that’s just not realistic.
On the reverse, you can’t reasonably expect 34 million impressions on a text post on Facebook, because the rules of what makes a Facebook post popular is entirely different.
All this to say: it’s so easy to get caught up in your work in a vacuum and forget the environment that you’re working in. It’s so easy, that you see people thinking they can remake the internet if they just get enough people on board, or make the Next Big Thing if only they get enough attention, without ever realizing what the environment they are working in actually allows or desires to work.
To use one of my least favourite phrases:
In conclusion: all data, and insights derived from that data, and even actions derived from those insights, can only be helpful by making sure all the data is Realistic, Simple, and Fun.