dlakelan

dlakelan, 4 days ago

For the non math nerds, you can imagine two groups, one billionaire has wealth growing like exp(t) and the rest of society has growth growing at a just marginally slightly lower rate exp((1-epsilon) t). Then after a while, the billionaire will own essentially 100% of everything.

For the math nerds out there, multiply by 1 in the form of exp(-t) / exp(-t) you get

1 / (1 + exp((1-epsilon - 1)t))
= 1/ (1+exp(-epsilon * t))

for large t exp(-epsilon * t) goes to zero and the result is 1/1 = 1

dlakelan, 3 days ago

@GhostOnTheHalfShell @feld @ChrisMayLA6

Ideally what you'd do is you'd develop a geographically local measure of the dollars/year you need to spend to maintain a kind of baseline living, and then you'd form a dimensionless ratio of I* = household_income / cost_of_basic_living... you could then ask questions about the value of I* (roughly "income") and the value of ( I* - 1 ) (roughly "disposable income").

I did develop some maps trying to measure these things but had technical difficulties

dlakelan, 3 days ago

@GhostOnTheHalfShell @ChrisMayLA6

The process involved downloading every record in the American Community Survey microdata, forming various samples of households through time, and fitting a Bayesian model with something like 30,000 parameters. This would take literal hours of computing time, and wouldn't always converge. There was no funding so I had to abandon the project. When it did converge it always gave the same answer, so the maps were probably pretty reasonable.

dlakelan, 3 days ago

@GhostOnTheHalfShell @ChrisMayLA6

Roughly the most important issues were

1. time and money
2. difficulty of fitting 30,000 parameter models
3. lack of technology for diagnosis of very large bayesian models
4. ambitiousness of the project

data management was actually not too bad. Mariadb had a method for searching CSV files as if they were tables, so I was able to use that for slicing and dicing.

dlakelan, 3 days ago

@GhostOnTheHalfShell @ChrisMayLA6

If I remember correctly I was working at the level of "public use microdata areas". These are geographic regions of roughly 100k people. So there are 350M/100k = 3500 of them. each one involved estimating a nonlinear function of household composition (number of people and their age), so roughly 5-10 parameters for that function, and then tying them together via regional relationships... so that's where roughly 30,000 parameters comes from.

dlakelan, 3 days ago

@GhostOnTheHalfShell @ChrisMayLA6

Diagnosis means roughly figuring out why Markov Chain Monte Carlo either didn't converge, or if it did converge whether the results "made sense" so that the estimates of the parameters were reliable.

dlakelan, 3 days ago

@GhostOnTheHalfShell @ChrisMayLA6

Iterative development of the model involves fitting the model, figuring out whether there are important considerations it fails to address, re-defining the model, and re-fitting it.

Roughly, fitting it might take 4 hours of computing say. So you get maybe 2 iterations of this process a day. Sometimes there are simple software bugs, sometimes there are "logical bugs". There have been a number of important methodological advances since 2017 or so.

dlakelan, 3 days ago

@GhostOnTheHalfShell @ChrisMayLA6

Some of those advantages could give you "rough fits" in perhaps 10-20 mins or something, so you could iterate much more quickly today potentially.

dlakelan, 3 days ago

@GhostOnTheHalfShell

It's absolutely wonderful, I would love to do this full time. Roughly the ideal situation would be something like $500k a year to fund me, a PhD Econ friend, and 3 students with an undergrad degree, one in an engineering or physics or biophysics discipline, one in a CS discipline with experience in databases, and one in an econ discipline.

dlakelan, 3 days ago

@GhostOnTheHalfShell

Probably a lot we could do. The obvious way to parallelize it just gives you say 5x as many MCMC samples from 5 different chains in the same 4 hour run. Which is great for final results, but bad for iterative development.

But like I said, a lot has changed since then. One thing is I now work in Julia and it is truly a blessing for this kind of stuff.

dlakelan, 3 days ago

@GhostOnTheHalfShell

A number of reasons, one of which is ideally we'd do things like food budget as a function of age, also we'd do heating and cooling budget as a function of location and climate. You want to estimate minimal budgets. So you don't just want to look at what people actually spend, because some of that is "disposable income".

dlakelan, 3 days ago

@GhostOnTheHalfShell
The markov chain processes the entire dataset at each iteration. It's quite intensive. But there are reasons you might be able to do regions in parallel and sacrifice some minor amount of information (costs in say Fresno don't really inform costs in say Kansas City or in Atlanta that much).

So, yeah, it's a real honest to goodness research project that roughly something like NSF should fund for 4 years.

dlakelan, 3 days ago

@GhostOnTheHalfShell

One of my motivations for the whole project was to understand the differences between rural and urban regions and how stark those were likely to be in CA.

dlakelan, 3 days ago

@GhostOnTheHalfShell

It's a good question. Usually grants are not available to any organization that isn't a 501c3 nonprofit, so there's a chicken-and-egg issue. I could create a 501c3 and try to get funding for it, but the organizations funding things prefer existing track records rather than new orgs... the overhead of forming a 501c3 isn't insane, but it's not zero either. You and I think the topic is very compelling, but it's directly opposed to an established Econ power structure

dlakelan, 3 days ago

@GhostOnTheHalfShell

Probably the best way to go about it is to get some kind of "preliminary" grant to show that the project is kind of possible and compelling and that the organization can handle it. But even to realistically get started we're talking $120k. That's what a CS undergrad earns in the LA area.

dlakelan, 3 days ago

@GhostOnTheHalfShell

You might enjoy diving into some preliminary analysis at the level of just slicing and dicing the ACS and making plots.

If you're interested I could mentor on that. I'd recommend installing Julia on a Linux box with 16 or 32 Gigs of RAM and a terabyte SSD, and setting up a github project... I can't really do the multi-hour-per-day to do the projects right now, but I could absolutely coach.

dlakelan, 3 days ago

@GhostOnTheHalfShell
It's enough to get started to see whether you find this stuff compelling. I'd say rather than doing mariadb and SQL queries you'd be doing more by hand filtering in Julia but it's still enough to know whether you want to do this stuff or not.

Start by making a GitHub project on the web with just a README and then cloning the empty project to your Mac. Easiest way

dlakelan, 3 days ago

@GhostOnTheHalfShell
Your biggest issue is running MariaDB on the same machine and having it take up RAM. It's very reasonable for a 32GB machine, doable with 16 but maybe not a great idea with 8GB.

Your best bet is a CONNECT table https://mariadb.com/kb/en/connect-csv-and-fmt-table-types/

dlakelan, 2 days ago

@GhostOnTheHalfShell
The less traditional stats you know the better 😉.

Don't add the datasets to the git repos, too big! But a script to do the download and extraction would help document what we downloaded and help anyone who wants to replicate.

git add, git commit, git push, and git pull are the most important getting started commands.

Did you get VSCodium and the Julia extension? I'll write a quick getting started notebook tomorrow you can use as a template for sorting and plotting

dlakelan, 4 days ago

@jackofalltrades
Now you can see why I keep thinking I need to start a nonprofit and get funding for research because this kind of question is the kind of question I want to ask and start to answer... Know anyone looking to fund nonprofit research into these questions?
@FantasticalEconomics

dlakelan, 4 days ago

@FantasticalEconomics @GhostOnTheHalfShell

There's a lot of talk about capitalism this, and socialism that. But most people hear "businesses owned by non-govt people" vs "businesses run by an authoritarian govt like in USSR".

Those are the two options people think we have, and so understandably they're not in favor of the second one... If we could explain better to everyday people we would get farther. Not only are those not the only options we don't even have either one right now.