When does damage quantization and rounding apply, and what parts are quantized and rounded?

[PublikDomain]

Damage quantization is an interesting concept and I'm finally getting around to applying it within my own spreadsheets. However, it is unclear to me when and where quantization applies, and when and where values are rounded.

Take an unmodded Hate for example. It deals 230 damage, so it has a damage quanta of 14.375. Its 34.5 Puncture damage is divided by the quanta to equal 2.4, rounded down to 2, and then multiplied back up to 28.75. So with how its base damage is spread out Hate actually deals 215.625 total damage. But what about damage from a Heavy Attack or a headshot? The wiki states that:

Quote

Damage mods such as Hornet Strike, Faction mods such as Bane of Grineer, and any other multipliers only multiply final quantized values and do not affect the quanta or damage composition.

Does it not affect the damage composition because there is a specific order of operations, or does it not affect damage composition because the math doesn't care what the order of operations is? For example, a Heavy Attack deals 6x damage for Scythes, so Hate's damage quanta would now be 86.25. Dividing its 207 Puncture damage by that new quanta still gives 2.4 which is still rounded down to 2 and scaled back up to 172.5, which is the same as its quantized 28.75 damage being calculated first and then scaled up by 6x after. So I'm assuming it's the latter: the math just works out this way.

A unmodded Hate's Heavy Attack that crits would deal 15x the total quantized damage (6 from the heavy attack multiplier * 2.5 from the crit), or 3,234.375. And if you test ingame it shows up as 3,235 damage. But why does it round up to 3,235 and not down to 3,234 which is the closer integer?

Since Hate is a Scythe it forces a Slash proc on Heavy Attack. 35% of a critting 3,235-damage hit is 1132.25, but the proc actually ticks for 1,208. This lines up with the unquantized 230 base damage: 230 * 6 * 2.5 * 0.35 = 1,207.5 which rounds up to the 1,208 observed ingame. What other effects are not based on the quantized damage?

When else does rounding occur? Is it only at the very end when damage numbers are displayed, or are there steps where a rounded value should be considered? Is the rounded integer value shown onscreen used for other other calculations or is it purely cosmetic? I'm noticing when trying to calculate Harmony's damage numbers that my real attacks are dealing damage close to the on-paper value but still off by a very slight amount. When I take out the big obvious numbers I'm left with a number like 9.005, which is almost the 9x damage I'm expecting but there's that extra bit left over. I can provide more examples tomorrow if the answers to my other questions don't get me there.

Thanks!

Edited June 30 by PublikDomain

[PublikDomain]

I came across this reddit post regarding status distribution and quantization: https://old.reddit.com/r/Warframe/comments/1au66z1/why_does_de_design_weapons_with_stats_that_wont/

While it doesn't quite answer my question about status distribution directly it does point out that the Galatine Prime's damage distribution causes it to never deal any Impact or Puncture damage. Because the Impact and Puncture damage are each 2.5% of the total damage, they are less than half of the quantum and thus get rounded down to 0.

Quantum: 280 / 160 = 17.5

Slash: 266 / 17.5 = 15.2, which rounds down to 15, and then scales back up to 262.5.

Impact/Puncture: 7 / 17.5 = 0.4, which rounds down to 0, and then scales to 0.

When you hit a Charger with an unmodded Galatine Prime, it will deal some multiple of 393.75 depending on your stance's hit multiplier.

So I tested this by equipping some Status Chance, Status Duration, and Attack Speed mods and used the block combo of Rending Crane to whack an Auditor. I was unable to inflict a single Impact or Puncture proc despite hitting it for over 6 minutes straight. At 3 hits per 2 seconds that's over 540 hits. According to the Arsenal, I should have had a 2% chance on each hit to deal Impact or Puncture which would have manifested as about 11 procs. So...

Status Chance is based on quantized damage distributions.

And the Arsenal's Status Chance breakdown is incorrect.

However, this also highlights an interesting property. If you follow the wiki it gives this example:

Quote

• The value of the example quanta will still be 100 / 16 = 6.25. Elemental and Physical bonuses do not affect the quanta.
• If Maim was used, for example, the Slash bonus calculates seperately using the base portion of slash, and adds to the total: round( 40 × 1.2 ÷ 6.25 ) × 6.25 = 50.
• The damage distribution would then be 31.25 Impact, 31.25 Puncture, and 37.5 + 50 = 87.5 Slash.

Replacing this with the Galatine Prime's stats we'd expect this:

Quote

• The value of the example quanta will still be 280 / 16 = 17.5. Elemental and Physical bonuses do not affect the quanta.
• If Heavy Trauma was used, for example, the Impact bonus calculates seperately using the base portion of Impact, and adds to the total: round( 7 × 0.9 ÷ 17.5 ) × 17.5 = 0.
• The damage distribution would then be 0 Impact, 0 Puncture, and 262.5 Slash.

IE, unless you add enough Impact damage to overcome half the quantization and round up in the rounding step, Impact and Puncture mods do not actually increase the damage of the Galatine Prime. And in practice this actually turns out to be true! Sort of...

An unmodded Galatine Prime with Rending Crane hitting a Charger deals 1,575 damage.

• With +15% Impact Damage it deals 1,575 damage
• With +20% Impact Damage it deals 1,575 damage
• With +30% Impact Damage it deals 1,645 damage and can now proc Impact
• With +35% Impact Damage it deals 1,645 damage
• ...
• With +120% Impact Damage it deals 1,645 damage
• ...
• With +210% Impact Damage it deals 1,645 damage
• ...
• With +285% Impact Damage it deals 1,715 damage

Following the wiki's example I should only expect to start dealing Impact damage at +125% Impact Damage:

Quote

• The value of the example quanta will still be 280 / 16 = 17.5. Elemental and Physical bonuses do not affect the quanta.
• If Collision Force was used, for example, the Impact bonus calculates seperately using the base portion of Impact, and adds to the total: round( 7 × 1.2 ÷ 17.5 ) × 17.5 = 0.
• The damage distribution would then be 0 + 0 = 0 Impact, 0 Puncture, and 262.5 Slash.

• The value of the example quanta will still be 280 / 16 = 17.5. Elemental and Physical bonuses do not affect the quanta.
• If R3 Collision Force + R2 Heavy Trauma was used, for example, the Impact bonus calculates seperately using the base portion of Impact, and adds to the total: round( 7 × (0.8 + 0.45) ÷ 17.5 ) × 17.5 = 17.5.
• The damage distribution would then be 0 + 17.5 = 17.5 Impact, 0 Puncture, and 262.5 Slash.

Instead what we see is that somewhere between +20% and +30% Impact Damage we overcome the quantization and start to round up. And if we change the order of operations a bit we find the culprit:

Quote

the Impact bonus calculates seperately using the base portion of Impact, and adds to the total

Instead what it really does is calculate the bonus with the base portion of Impact as one step.

Instead of round(7 / 17.5) * 17.5 + round(7 * 0.3 / 17.5) * 17.5 as two steps, it's round(7 * (1 + 0.3) / 17.5) * 17.5.

So the wiki is wrong too.

Plugging in the previous example Impact Damage percentages into my spreadsheet calculator I get this:

• With +15% Impact Damage it should deal 1,575 damage and it does
• With +20% Impact Damage it should deal 1,575 damage and it does
• With +30% Impact Damage it should deal 1,645 damage and it does
• With +35% Impact Damage it should deal 1,645 damage and it does
• ...
• With +120% Impact Damage it should deal 1,645 damage and it does
• ...
• With +210% Impact Damage it should deal 1,645 damage and it does
• ...
• With +285% Impact Damage it should deal 1,715 damage and it does

For Elemental damage and elemental mods, the wiki is also wrong in the same way. Base elemental damage and elemental mods are combined and quantized in one step as well, not quantized separately and then added.

As an example, the Trumna deals 82 damage and has a quantum of 5.125. It deals 53 of its base damage as Heat. If you add in a R0 Hellfire for +15% Heat you should expect to get this:

Heat Damage = round(53 / 5.125) * 5.125 + round(82 * 0.15 / 5.125) * 5.125 = round(10.341...) * 5.125 + round(2.487...) * 5.125 = 51.25 + 10.25 = 61.5 Heat Damage

This would combine with the quantized 30.75 Impact damage for 92.25 total damage on hit. Against a Charger with weakness to Heat we should expect to deal 123 damage exactly. Instead the Trumna hits for 131. If the Heat damage is instead done in one step we would expect this:

Heat Damage = round((53 + 82 * 0.15) / 5.125) * 5.125 = round(12.741) * 5.125 = 66.625 Heat Damage

Which would increase by 1.5x and add to the 30.75 quantized Impact Damage to hit our Charger for 130.6875 damage, which rounds up to the 131 we observe.

So the correct way to quantize IPS and Elemental damage is:

IPS Damage = round(Base IPS Damage * (1 + IPS Damage Mods) / Quantum) * Quantum

Elemental Damage = round((Base Elemental Damage + Base Damage * Elemental Damage Mods) / Quantum) * Quantum

Which I think answers all of my questions!

Edited Friday at 02:35 PM by PublikDomain

[RLanzinger]

To @PublikDomain,

First, there was a few UI bug that do not display the correct used damage. Display is wrong but Damage calculated and used are correct.

Second, Damage Quanta in wikia is wrong as base on some loosy post on REDDIT. Damage quanta IS 16 pts of damage IE damage are not diminish by whole value but per 16 damage then MODULO 16. This is what make the dude think that damage was rounded to 1/16th...

• This mean, in case of ONE arrow damage 2 consecutive targets, then first loose a multiple of 16 damage and the rest is applied to the second target.

Third, TOTAL Damage are NOT damage(s)... What the game do is damage per TYPE (Impact, Slash, Puncture... +the 7 elements) then display you the Total SUM of it.

• This mean what YOU see is sum of a lot of little tiny calculation but not WHAT HAPPENS.

 

So the TOTAL DAMAGE in Game UI from the Arsenal is not exact to the point of Damage but give you a close approximation of what happens. Nothing more, Nothing less...

This is why Simulacrum exist but even the Damage display is a SUM not ALL of what happens.

[PublikDomain]

8 hours ago, RLanzinger said:

Second, Damage Quanta in wikia is wrong as base on some loosy post on REDDIT. Damage quanta IS 16 pts of damage IE damage are not diminish by whole value but per 16 damage then MODULO 16. This is what make the dude think that damage was rounded to 1/16th...

• This mean, in case of ONE arrow damage 2 consecutive targets, then first loose a multiple of 16 damage and the rest is applied to the second target.

Just to be clear, I'm not asking about damage reduction on punchthrough. Does that use a similar system?

I'm asking about direct dealt damage like with a melee weapon and how it appears vs how it's applied. For example, if you take an unmodded Hate and heavy attack a Deimos Charger you get this:

If the damage is quantized the way it's explained on the wiki:

... then this all works out. I'm not dealing Hate's listed 1,380 Heavy Attack damage, I'm dealing the quantized 1,293.75 damage. Against a regular Charger with weakness to Slash it again works out to show this:

Where the 50% weakness to Slash is applied to the quantized damage resulting in 1,768.125 damage. Which rounds down.

But if you crit, you deal this:

The quantized total damage should be 4,420.3125 and as before round down to 4,420. But instead it rounds up to 4,421?

A lvl41 regular Charger should have 4,563.73 HP. If it's taking the quantized 4,420.3125 damage then it should survive a crit heavy attack. If it's actually taking the unquantized 4,657.5 damage then it should not survive.

It survives:

... by a teeny-tiny sliver. So it can't be taking the unquantized damage, and quantization is not purely cosmetic.

And regardless of which Charger was hit both regular Heavy Attacks non-crit attacks produced a Slash proc which ticked like this:

If this proc were 35% of the quantized damage I actually dealt, it would tick for 452.81. If it's based on the listed 230 base damage then it makes sense: 230 * 6 * 0.35 = 483.

So I'm again left with the following conclusions:

1. Damage quantization is not purely cosmetic, and does actually affect the damage dealt to enemies.
2. DoT proc damage is not based on quantized total damage, but instead on the unquantized base damage.
3. Rounding of visual values is inconsistent.

Where here:

Should the modulo you mention be?

Edited June 30 by PublikDomain

[RLanzinger]

To @PublikDomain :

The principle is a 4bit system (4 bit = 16 in decimal).

Quantitized Damage is not 1,380 / 16 = 86,25 . It's 1,380 % 16 = 86 x 16 +4, that mean the system Do HP - 16, 86 times if necessary then HP - 4.

That for faster calculation, the quanta mean the system make subtraction on the 4bit instead of first bit.

 

Also the explanation on Wikia, is a analysis made by a player. Its not HOW [DE] explain it...

If you want to know how the game does and be sure that what happens : Reverse Engineering the game Or ASK [DE];

 

anything else is a ingenious guess...at best.

[PublikDomain]

1 minute ago, RLanzinger said:

Also the explanation on Wikia, is a analysis made by a player. Its not HOW [DE] explain it...

If you want to know how the game does and be sure that what happens : Reverse Engineering the game Or ASK [DE];

 

anything else is a ingenious guess...at best.

Where has DE explained how this works? I've love a link.

1 minute ago, RLanzinger said:

Quantitized Damage is not 1,380 / 16 = 86,25 . It's 1,380 % 16 = 86 x 16 +4,

Where does this then get involved with damage types?

Hate's Heavy Attack deals 207 Impact + 207 Puncture + 966 Slash damage, or 1,380 total.

How is that affected by quantization to produce the ingame 1,294 damage against an enemy with no weaknesses or resistances and the ingame 1,768 against an enemy with a weakness to Slash?

[PublikDomain]

So as a little update I've got quantization figured out. Right or wrong as RLanzinger says, the rounding behaviour described by the wiki is accurate in practice. So I'll keep using it. To answer some of my original questions:

1. What is quantized? Base damage is quantized, and so is elemental damage like from mods or effects. These are quantized separately and then added together. Order of operations for scaling by damage mods, faction damage mods, etc. do not matter in the order but are more convenient to manipulate if they are left to the end.
2. How does rounding work specifically? I still don't know.
3. What about my 9.005? The answer is crit quantization, not rounding!

---

One thing I found that I had also missed was crit quantization. Critical Damage is quantized too!

So if you have a Critical Damage of 2.2, you actually have a Critical Damage of 2.2036630036630036630036630036636... This only really shows up when you have very big critical damage numbers and accounts for the little bit of fudge I was seeing.

---

Regarding elemental damage quantization mentioned above, like for a 90% damage mod, the amount of damage follows the same formula as base damage values:

But where "Total Damage Type Value" is instead Total Unquantized Base Damage x Elemental Percentage. So if a weapon deals 100 base Heat damage and you add a 90% Heat mod:

1. Quantum is 100 /  8 = 12.5.
2. The base 100 Heat damage quantizes into 12.5 * round(100 / 12.5) which just equals 100.
3. The 90% Heat mod quantizes into 12.5 * round((100 * 0.9) / 12.5) which equals 87.5.
4. The total Heat damage is 100 + 87.5 = 187.5.

If you shot an enemy without any weaknesses or resistances, you would see either 187 or 188 depending on how it rounds the display value.

---

Now I'm left with another question. Say that I have a Veldt with a 120% Gas damage and 60% Radiation damage. It has the following damage breakdowns:

Before Quantization:

• 23.4 Puncture (9.29%)
• 23.4 Impact (9.29%)
• 54.0 Radiation (21.43%)
• 43.2 Slash (17.14%)
• 108.0 Gas (42.86%)

After Quantization:

• 22.50 Puncture ( 8.89%)
• 22.50 Impact ( 8.89%)
• 56.25 Radiation ( 22.22%)
• 45.00 Slash ( 17.78%)
• 106.88 Gas ( 42.22%)

The Arsenal breakdown of status proc chances lines up with the pre-quantization values. But if the damage number on the Arsenal is wrong, maybe this is too? Which set of numbers are status proc ratios actually based on?

The difference is quite small either way but it would be nice to know for sure. For now I will assume that it actually does use the Arsenal's values.