aeva,
@aeva@mastodon.gamedev.place avatar

so I'm trying to work out what the exact flags cmake is setting for gcc for a debug build, and the docs just say "the usual ones", and my searches are saturated with stackoverflow posts that are topically adjacent but unfortunately irrelevant to my problem at hand because I'm trying to work out why my debug builds have horrible numeric issues or something but my release builds don't

and so I get the clever idea of looking at what cmake generated, and oh my god that is a lot of cmake

aeva,
@aeva@mastodon.gamedev.place avatar

shocking, I know, but trying to work out what exactly cmake is doing is not making me feel less stressed out.

aeva,
@aeva@mastodon.gamedev.place avatar

ah I see, CMakeCache.txt is the file that has what I want to know in it.

It's just passing in -g for debug builds and -O3 for release builds. sigh I guess that's a reasonable thing to do.

aeva,
@aeva@mastodon.gamedev.place avatar

-O3 being the thing that seems to make it work is a bit foreboding

tastytronic,

@aeva 😱

aeva,
@aeva@mastodon.gamedev.place avatar

ok this is seriously haunted. I narrowed it down to a simple test case. I have a SDF that is just a cube function with no transform applied.

If I just eval it at the start, I get this if it's a release build:

Eval(vec3(0, 0, 1)) == 0

And if I do the same for a debug build I get:

Eval(vec3(0, 0, 1)) == -1

The correct value in this case is 0, so that's super weird.

If I try some other known surface points it'll give me the correct answer though, but if it's not z=-1 or z=1

aeva,
@aeva@mastodon.gamedev.place avatar

stepping into this mess with the debugger in a debug build is just too much "zero cost" abstraction to step through because I'm using GLM. So I put in some print statements. First step is transforming the point into the shape function's local space. The shape is at the world origin, so the transform is an identity matrix.

Applying that transform to point (0, 0, 1) somehow returns (0, 0, 0) !!!!!!!!!!

aeva,
@aeva@mastodon.gamedev.place avatar

forcing my program to run single threaded doesn't help, so I think I can probably rule out spooky concurrency stuff at least.

IDK maybe there's just too much "zero cost" going on in glm and gcc fucks up the codegen when its not all optimized out? I certainly don't remember seeing MSVC have any problems with it.

TomF,
@TomF@mastodon.gamedev.place avatar

@aeva I mean obviously it's not actually "zero" cost, but the harder you work, the closer to zero it becomes.

aeva,
@aeva@mastodon.gamedev.place avatar

@TomF the cost is fucky long callstacks in debug builds

agersant,
@agersant@mastodon.gamedev.place avatar

@aeva you probably already thought about this but just in case: could there be missing initialization for some struct or variable and the undefined behavior kicks in differently for debug vs release. (Eg. Some field zeroed out vs not)

aeva,
@aeva@mastodon.gamedev.place avatar

@agersant as far as I can tell it seems to be constructing fine. The contents of the matrix being multiplied is as expected:
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1

aeva, (edited )
@aeva@mastodon.gamedev.place avatar

Ok I narrowed it down, but I'm no less baffled.

vec4 tmp1 = Identity * vec4(0, 0, 1, 1);
... produces vec4(0, 0, 1, 1)

vec3 tmp2 = tmp1.xyz;
... produces vec3(0, 0, 1)

vec3 tmp3 = tmp2 / vec3(tmp1.w)
... produces vec3(0, 0, 0)!!!!!!

but only in debug builds. wtf

mcc,
@mcc@mastodon.social avatar

@aeva What is tmp

aeva,
@aeva@mastodon.gamedev.place avatar

@mcc a typo

mcc,
@mcc@mastodon.social avatar

@aeva okay. Here's a silly question. If you take that code and you put it in its own program. Apart from all the tangerine stuff. Do you see the same thing.

I'm wondering if something that appears unrelated could be in fact related (worst case scenario stack/heap corruption, which could be accidentally avoided in debug builds by different offsets). Or if there could be a step where data is transferred between stack frames in a way your post doesn't have rlomt k convey.

scottmichaud,
@scottmichaud@mastodon.gamedev.place avatar

@mcc @aeva Only thing that stands out to me -- what is the dimension(s) of Identity?

aeva,
@aeva@mastodon.gamedev.place avatar

@scottmichaud @mcc identity is a 4x4

aeva,
@aeva@mastodon.gamedev.place avatar

@mcc the math in isolation does the right thing in both build configs :|

this is the function that's messing up https://github.com/Aeva/tangerine/blob/10def3cfe7fd1c27a0bcd32c574bfd7b994f4128/tangerine/sdf_evaluator.cpp#L235

In optimized builds it is fine, but in debug builds the return value is wrong.

Stack weirdness sounds plausible, though I'm not sure what I'd do about that if its spilling because the codegen is too verbose or something

mcc,
@mcc@mastodon.social avatar

@aeva And… just checking, I assume you have tried replacing "Tmp.www" with "Tmp.w"?

aeva,
@aeva@mastodon.gamedev.place avatar

@mcc glm is fussy about the type mismatch but it does the same if i change it from Tmp.www to vec3(Tmp.w). I don't think it accepts somevec3 / somefloat.

mcc,
@mcc@mastodon.social avatar

@aeva I'm looking over the GLM manual trying to find an explicit definition of multiplication of two vec3s. I'm bothered I can't seem to find it

mcc,
@mcc@mastodon.social avatar

@aeva Can you humor me, and attempt

(1/Tmp.w)*Tmp.xyz

Surely it allows left-side scalar multiplication, at least

aeva,
@aeva@mastodon.gamedev.place avatar

@mcc sure one sec lemme see if I can break it again.

Somehow adding target_compile_options(tangerine PRIVATE -std=c++17) to the cmake thingy and changing nothing else makes the problem go away ._.

mcc,
@mcc@mastodon.social avatar

@aeva CURSED !!!!

aeva,
@aeva@mastodon.gamedev.place avatar

@mcc wait no false alarm, I'm just tired and am making silly mistakes. 1 sec

aeva,
@aeva@mastodon.gamedev.place avatar

@mcc ok good news I reverted the "what if I just remove the w divide" work around that I added confirmed and then promptly forgot about and it's broken again. gonna try the thing you suggested now

aeva,
@aeva@mastodon.gamedev.place avatar

@mcc ok (1/Tmp.w)*Tmp.xyz results in an error message that is long enough that my terminal emulator craps out, but it looks like it's a template arg type deduction/substitution failure.

vec3 Tmp3 = float(1.0 / Tmp.w) * Tmp2;

prevents the error but now instead of occasionally getting some values right it gets all of them wrong and no verts are generated

sorry mismatched names of "tmp" between the example and this

mcc,
@mcc@mastodon.social avatar
oblomov,
@oblomov@sociale.network avatar

@aeva @mcc PROGRESS!

aeva,
@aeva@mastodon.gamedev.place avatar

@mcc oh I made a silly mistake again. 1 sec

aeva,
@aeva@mastodon.gamedev.place avatar

@mcc ok your proposed change works perfectly

aeva,
@aeva@mastodon.gamedev.place avatar

@mcc I am very interested in any theories you have as to why this might be

mcc,
@mcc@mastodon.social avatar

@aeva I think @tess had the right of it, that the language magic involved in making the swizzles work may have been valid C++ but hit a GCC bug. It's possible the problem would have gone away if you'd upgraded your compiler.

I would recommend filing a bug on GLM, although that might be hard if your attempts to make a more minimal use case have not succeeded.

aeva,
@aeva@mastodon.gamedev.place avatar

@mcc @tess yeah so far I can't repro it in isolation

mcc,
@mcc@mastodon.social avatar

@aeva @tess since tangerine is open source maybe you could just point them at a commit and give them building instructions

oblomov,
@oblomov@sociale.network avatar

@mcc @aeva @tess
might be too big, but maybe creduce manages to get something reasonable out of it.

tess,
@tess@mastodon.social avatar

@aeva @mcc I suppose it's silly to ask if there's any chance your LastFoldInverse is horked from a different operation?

Also, have you tried breaking out the operations, something like this?

 vec3 Tmp2 = Tmp.xyz;  
 Tmp2 /= Tmp.www;  
 return Tmp2;  
aeva,
@aeva@mastodon.gamedev.place avatar

@tess @mcc this variation also works.

Also in both this and left multiply by rcp, it fixes only some of my models without an obvious pattern :| likely because there are more surprises like this elsewhere

tess,
@tess@mastodon.social avatar

@aeva @mcc yeah I guess I would probably play it safe and avoid using more than one swizzle per line of code? I dunno. Clearly between the library and the compiler there is something bad happening.

mcc,
@mcc@mastodon.social avatar

@aeva @tess What about upgrading GCC? Is that possible?

tess,
@tess@mastodon.social avatar

@mcc @aeva how old a version we even talking about? I saw these kinds of issues like 15+ years ago. I would have assumed they'd been resolved by now.

If the compiler version is fairly new I'd say the solution isn't "get latest"; it's "switch to LLVM".

aeva,
@aeva@mastodon.gamedev.place avatar

@tess @mcc
I just have whatever GCC version is packaged in Debian Testing, which I should hope is current. gcc -v says it's "gcc version 12.2.0 (Debian 12.2.0-14)"

aeva,
@aeva@mastodon.gamedev.place avatar

@tess @mcc I'm down for switching to clang, but it seems that clang is missing std::filesystem sigh

I don't suppose msvc can cross compile to linooks ._.

TomF,
@TomF@mastodon.gamedev.place avatar

@aeva @mcc
<loads pistol>
Compiler's haunted.

oblomov,
@oblomov@sociale.network avatar

@aeva @mcc would it even be possible that this was due to some obscurely missed dependency that caused debug to be linked against release files?

Specialist_Being_677,
@Specialist_Being_677@hachyderm.io avatar

@aeva @mcc honestly that might be the over alignment for simd thing... Also there is a separate variable and property to choose standard version, something like CXX_STANDARD_VERSION

aeva,
@aeva@mastodon.gamedev.place avatar

@mcc nobody really knows how glm works. I think it just appeared one day.

mcc,
@mcc@mastodon.social avatar

@aeva It's just sort of logically implied by the universe, like prime numbers.

tess,
@tess@mastodon.social avatar

@aeva @mcc after looking at both the GLM documentation and source code: this tracks

Holy shit, www appears to be them using macros to implement a poor man's version of extension methods from C#; this is EXTREMELY CURSED

(Also, vecN appears to support /= with a scalar, so you could just do Tmp /= w; return Tmp?)

tess,
@tess@mastodon.social avatar

@aeva @mcc I mean, it's also likely they have defined scalar operator/ in some equally absurd sort of way, so honestly I'd just go for it and let the compiler tell you know.

tess,
@tess@mastodon.social avatar

@aeva @mcc but based on 20+ years of dealing with gcc I would say that it is almost certainly some sort of cursed interaction of a macro referencing the same object several times in the middle of a return statement, interacting with local destructors in a way that's not technically incorrect under C++ rules but also extremely unintuitive*

  • by which I mean "wrong for any sensible definition of wrong created by a human being who has read the C++ spec, but not to the kinds of people who code gcc"
tess,
@tess@mastodon.social avatar

@aeva @mcc the fact that compiler optimization seems to eliminate the issue adds some strength my conviction on this; the ordering (or even execution) of destructors/memory cleanup would depend heavily on what kind of build configuration you were using.

aeva,
@aeva@mastodon.gamedev.place avatar

@tess @mcc I updated my isolated test case to include a return, but it doesn't reproduce there.

tess,
@tess@mastodon.social avatar

@aeva @mcc I mean frankly, I'm also riding with andi on the "what the hell does dividing two vec3s do?" train; it really feels like you should be dividing by a scalar

xgranade,
@xgranade@wandering.shop avatar

@tess @aeva @mcc I'll third that; I don't know what that means in terms of linear algebra other than possibly the Hadamard (aka pointwise) product?

tess,
@tess@mastodon.social avatar

@xgranade @aeva @mcc it's clear glm is from the OVERLOAD ALL THE OPERATORS EVEN IF IT MAKES NO SENSE era of C++, but the net result of that is that a lot of them end up just kind of being mystery operations

mcc,
@mcc@mastodon.social avatar

@tess @xgranade @aeva I mean I think probably it does whatever glsl does, which might not necessarily be a coherent linear algebra operation. But I do wish it were explicitly in in the documentation.

aeva,
@aeva@mastodon.gamedev.place avatar

@mcc @tess @xgranade it is supposed to do what GLSL does. vec3 / vec3 is componentwise divide

aeva,
@aeva@mastodon.gamedev.place avatar

@mcc @tess @xgranade so it's equiv to vec3(x1 / x2, y1 / y2, z1 / z2)

aeva,
@aeva@mastodon.gamedev.place avatar

@mcc @tess @xgranade except when spooky stack ghosts do spooky things or something

xgranade,
@xgranade@wandering.shop avatar

@tess @aeva @mcc Makes a lot of sense. There can be some use for like SIMD processing, but yeah, needs to be documented.

tess,
@tess@mastodon.social avatar

@xgranade @aeva @mcc I created a string library in middle school where unary + was "to upper case" and / was "remove all occurrences of the second operand from the first".

It was extremely stupid but I felt very clever.

xgranade,
@xgranade@wandering.shop avatar

@tess On one hand, that's not much worse than << and >> being relabeled the "stream insertion" and "stream extraction" operators. Of course, I'm also the fool who overloaded &amp; not to mean bitwise AND but tensor products.

tess,
@tess@mastodon.social avatar

@xgranade to be fair, << has the benefit of being lower precedence than all ref/deref and arithmetic operators, and more or less make sense in context.

They were also added way before variadic templates, which is how you do that sort of thing today (though in some ways they are more elegant).

The thing that was gross about my library wasn't so much the operators, but the ring buffer it used for intermediates, so you could overrun it with too complicated of an expression.

ipd,

@aeva
Unh huh, and?

aeva,
@aeva@mastodon.gamedev.place avatar

@ipd seems to be a code gen bug w/ gcc, maybe

ipd,

@aeva
Hmmmm... interesting.. what else?

aeva,
@aeva@mastodon.gamedev.place avatar

@ipd glm does some diabolical template metaprogramming magic to make glsl's semantics work in c++, and for some reason the component-wise divide between two vec3s produces the wrong output with controlled inputs so idk. I haven't dug into how it expands because it's quite verbose for being a debug build.

ipd,

@aeva
You dug into the logs? How's this working out for you?

aeva,
@aeva@mastodon.gamedev.place avatar

@ipd I solved it. The way GLM emulates GLSL's swizzle syntax and/or the SIMD stuff does the wrong thing in debug builds. Easily worked around, just annoying to track down.

https://github.com/Aeva/tangerine/commit/8fecb0a75079d6128f85ad485415acce088265a5

TomF,
@TomF@mastodon.gamedev.place avatar

@aeva The usual culprit here is initiialisation. Debug builds often init to magic numbers like 0xCDCDCDCD and release builds default to "what is your security clearance, citizen?"

aeva,
@aeva@mastodon.gamedev.place avatar

@TomF the constructor is setting the matrix to the correct identity

lisyarus,
@lisyarus@mastodon.gamedev.place avatar

@aeva Maybe there are some defines that mess with constructors or smth? I recall quaternions with WXYZ order (or maybe XYZW) being completely broken because it was turned on by a macro and half of the lib supported it and the other half didn't.

aeva,
@aeva@mastodon.gamedev.place avatar

@lisyarus I have a header that sets up all the defines before including so that everything is coherent

lisyarus,
@lisyarus@mastodon.gamedev.place avatar

@aeva In my case it wasn't a coherence problem, all the code was in just one file. It's that half of the quaternion class implementation didn't support that macro at all :(

aeva,
@aeva@mastodon.gamedev.place avatar
gob,
@gob@mastodon.gamedev.place avatar

@aeva I tried to repro but can't (debug mode, gcc 11.3.0, latest git glm)

gob,
@gob@mastodon.gamedev.place avatar

@aeva i suspect there's some undefined values making it in the identity matrix or something, maybe fsanitize=undefined can help ?

aeva,
@aeva@mastodon.gamedev.place avatar

@gob the value should be set by the constructor, but it's worth a try

aeva,
@aeva@mastodon.gamedev.place avatar

@gob I wasn't able to repro in an isolated test case either. I'm not on the latest GLM, so that might be part of it. Some people have commented that I'm probably hitting a GCC bug

aeva,
@aeva@mastodon.gamedev.place avatar

I updated GLM since it was a few versions behind anyway, but the debug build behavior remains the same ._.

I was kind of hoping I could use optimization pragmas to selectively opt files or functions into -O3, but I wasn't able to get that working, and iterating on it is too tedious.

Maybe I should give up on GCC and switch to clang :/

ruby0x1,
@ruby0x1@mastodon.gamedev.place avatar

@aeva GLM_FORCE_CTOR_INIT maybe? at some point they stopped initializing by default, if you wanted that behaviour, you gotta force it back on. this leads to garbage in memory?

aeva,
@aeva@mastodon.gamedev.place avatar

oh good, this issue also repros w/ clang debug builds 😩

mcc,
@mcc@mastodon.social avatar

@aeva uhhh

If it reproduces with clang then possibly GLM really is doing something outside the language spec

tess,
@tess@mastodon.social avatar

@mcc @aeva yah that's my read too. (Also ew.)

What does the code in question look like if you expand all of the layers of macros?

If I can see it exploded I can probably formulate a theory as to what exactly is happening.

aeva,
@aeva@mastodon.gamedev.place avatar

@tess @mcc how does one do that?

tess,
@tess@mastodon.social avatar

@aeva @mcc by hand, usually. Unless you've got a very smart code editor.

Also there appear to be two different implementations of swizzle based on a value; you might try setting the other one in your makefile and see if that resolves your issues.

tess,
@tess@mastodon.social avatar
aeva,
@aeva@mastodon.gamedev.place avatar

@tess @mcc I think that ended up being the thing. I got there from a different direction though. Switching to GLM_FORCE_PURE to use constexpr instead of intrinsics also seems to opt into the swizzle function variant. I had to hack up a polyglot glsl file to compile w/ it, but it seems to all work now.

aeva,
@aeva@mastodon.gamedev.place avatar

ok! it turns out this was GLM's fault, probably. Experimenting with GLM_FORCE_PURE (replaces intrinsics w/ constexprs) also opted into the swizzle variant where swizzles are member functions. Some hax to update the polyglot glsl parts of the code base later, and now debug builds are doing the right thing.

fell,
@fell@ma.fellr.net avatar

@aeva This is really interesting. I would try and step through the assembly to see where it goes wrong.

Also, try disabling some GLM features like accessing the XYZ members as RGB. (there is a define for that)

aeva,
@aeva@mastodon.gamedev.place avatar

@fell that's going to be so verbose. the callstacks for unoptimized GLM code in the debugger are awful

fell,
@fell@ma.fellr.net avatar

@aeva I would try. It can't take that long.

Specialist_Being_677,
@Specialist_Being_677@hachyderm.io avatar

@aeva are any of the variables members of a class, and are the types getting used to build vectorized code?

(Over alignment for simd stuff needs annoying hacks or super new c++, and when it breaks it looks really hard to understand like this.)

aeva,
@aeva@mastodon.gamedev.place avatar

@Specialist_Being_677 if simd is happening it's news to me

Specialist_Being_677,
@Specialist_Being_677@hachyderm.io avatar

@aeva ok, the mention of optimization near a computer desk just reminded me, as I've personally had that problem a bunch of times and it's annoying to debug.

jalcine,
@jalcine@todon.eu avatar

@aeva man this makes me wince in cmake_commands.json heh

be,
@be@floss.social avatar

@aeva cmake --build build-dir -v will print the full commands as it builds

  • All
  • Subscribed
  • Moderated
  • Favorites
  • random
  • ngwrru68w68
  • rosin
  • GTA5RPClips
  • osvaldo12
  • love
  • Youngstown
  • slotface
  • khanakhh
  • everett
  • kavyap
  • mdbf
  • DreamBathrooms
  • thenastyranch
  • magazineikmin
  • megavids
  • InstantRegret
  • normalnudes
  • tacticalgear
  • cubers
  • ethstaker
  • modclub
  • cisconetworking
  • Durango
  • anitta
  • Leos
  • tester
  • provamag3
  • JUstTest
  • All magazines