Testing is probably my favorite topic in development and I kind of wish I could make it my "official" specialty but no way in hell am I taking a pay cut and joining the part of the org nobody listens to.
> "SQL should be the first option considered for new data engineering work. It’s robust, fast, future-proof and testable. With a bit of care, it’s clear and readable." (over polars/pandas etc)
SQL has nothing to do with fast. Not sure what makes it any more testable than polars? Future-proof in what way? I guess they mean your SQL dialect won't have breaking changes?
I’m also a duckdb convert. All my notebooks have moved from Pandas and polars to Duckdb. It is faster to write and faster to read (after you return to a notebook after time away) and often faster to run. Certainly not slower to run.
My current habit is to suck down big datasets to parquet shards and then just query them with a wildcard in duckdb. I move to bigquery when doing true “big data” but a few GB of extract from BQ to a notebook VM disk and duckdb is super ergonomic and performant most of the time.
It’s the sql that I like. Being a veteran of when the world went mad for nosql it is just so nice to experience the revenge of sql.
I personally find polars easier to read/write than sql. Especially when you start doing UDFs with numpy/et. al. I think for me, duckdb's clear edge is the cli experience.
> It is faster to write and faster to read
At least on clickbench, polars and duckdb are roughly comparable (with polars edging out duckdb).
I use them both depending on which feels more natural for the task, often within the same project. The interop is easy and very high performance thanks to Apache Arrow: `df = duckdb.sql(sql).pl()` and `result = duckdb.sql("SELECT * FROM df")`.
Author here. I wouldn't argue SQL or duckdb is _more_ testable than polars. But I think historically people have criticised SQL as being hard to test. Duckdb changes that.
I disagree that SQL has nothing to do with fast. One of the most amazing things to me about SQL is that, since it's declarative, the same code has got faster and faster to execute as we've gone through better and better SQL engines. I've seen this through the past five years of writing and maintaining a record linkage library. It generates SQL that can be executed against multiple backends. My library gets faster and faster year after year without me having to do anything, due to improvements in the SQL backends that handle things like vectorisation and parallelization for me. I imagine if I were to try and program the routines by hand, it would be significantly slower since so much work has gone into optimising SQL engines.
In terms of future proof - yes in the sense that the code will still be easy to run in 20 years time.
> I disagree that SQL has nothing to do with fast. One of the most amazing things to me about SQL is that, since it's declarative, the same code has got faster and faster to execute as we've gone through better and better SQL engines.
Yeah, but SQL isn't really portable between query all query engines. You always have to be speaking the same dialect. Also, SQL isn't the only "declarative" dsl, polars's lazyframe api is similarly declarative. Technically Ibis's dataframe dsl also works as a multi-frontend declarative query language. Or even substrait.
Anways my point is that SQL is not inherently a faster paradigm than "dataframes", but that you're conflating declarative query planning with SQL.
> Now the downside … because health care is free, everyone uses it and the wait times are longer. My grandfather recently required an MRI (non life threatening). The wait time in Ontario was 3 months. He drove to the USA, paid out of pocket, and had it done within in week …
Imo, singapore solves this well, by ensuring that some cost is borne by the patient at point of use, but it's never anything excessive. No one goes bankrupt from emergency hospital visits.
It'd be a very interesting thought experiment to completely redefine the economic and fiscal plan of the US.
It'd be interesting if we greatly reduced personal income taxes in the majority of cases. Removed corporate income tax (the idea being corporate income tax hurts all equity owners, whereas we specifically want to target the very wealthy), and then implemented a progressive wealth tax that kicks in starting at 7.5M or something (maybe something like 1.5% from 7.5M-15M, 2.0% from 15M-50M, and then 3% from 50M+, this could be done more smoothly if desired). We'd need to penalize capital flight with something (something like a 40% exit tax or something). The IRS would have to be greatly empowered to support this, net worth declarations would all be declared by the tax filer, and if audited and found to be incorrect, the IRS will purchase the asset at 70% of its declared value (or some other better mechanism can be thought of to ensure that NW is declared accurately). We'd also simplify the tax code in all other cases, removing all forms of credit/deductions/etc on income. This will free up IRS audit resources to focus on declared net worth fraud. Remove all forms of non-sin tariffs entirely.
Get rid of entitlements taxes and spend and replace it with a negative income tax that is calculated to provide a very base level of existence that never disincentives work.
Then remove government backed student loans, but make them treated the same as other loans (ie. students can declare bankruptcy). Kids should also get a 5k per annum payment that's held in trust by the government that's invested into the US total market that gets linearly released to them from the age of 18-30.
I also propose that we remove all consumption taxes that aren't a sin tax. Ie. keep taxes on alcohol, gas, etc. Implement a very high luxury goods tax.
Then completely change the medical system to be more similar to Singapore with mandatory copays of something like $20-50 (to prevent abuse), but also provide universal coverage with the government setting prices with pharma/providers.
Dissolve the ability of states to collect taxes and provide federal funding per head. Standardize/centralize all stuff like the DMV. Ie. have a single federal agency to de-duplicate work.
I don't think this stuff is viable at all (would require dramatic tweaks), but it'd be a very interesting thought experiment to see how something that tries to encourage business friendliness, the strength of the income earning population, and incentives capital effiency (over tax effiency) and investment does.
I really think that the fundamental market forces of capitalism are very powerful, we just need to change the incentives structure so that the benefits aren't captured strictly by the ultra wealthy.
>Removed corporate income tax (the idea being corporate income tax hurts all equity owners, whereas we specifically want to target the very wealthy),
I'd back this whole plan to a hilt but want to highlight this section.
I'm on board with eliminating corporate taxes under 1 condition: the company, its executives, and its board - are fully 100% banned from participating in politics in ANY way besides each individual citizen voting in normal elections.
Lobbying? Banned. PAC contributions? Banned. Campaign contributions? Banned. Hiring former government workers as "government liaisons" or "government relations" consultants? Banned.
If a corporation wishes to not pay taxes to the government, it has ZERO business involving itself with politics.
I work for a big company (Apple) but I have no idea who Frank is, nor how to sponsor them; and even if I knew them and how to sponsor them, the money would come directly from my pocket instead of Apple’s banking account.
If libsodium is useful to you, please keep in mind that it is maintained by one person, for free, in time I could spend with my family or on other projects. The best way to help the project would be to consider sponsoring it, which helps me dedicate more time to improving it and making it great for everyone, for many more years to come.
Frank does great work that is critical to many businesses, and should get funded to do it professionally.
However, donating money to an open collective is prohibitively hard for most big companies. Maybe the world should be different (or maybe not, since it would be easy for employees to embezzle money if they could direct donations easily), but that's how it works currently.
AFAICT, there is also no fiscal sponsor, so the donation matching suggested in a sister comment won't apply.
This is why Geomys (https://geomys.org) works the way it does, and why it has revenue (ignoring the FIPS and tlog sides of the business) which is 30-50x of some GitHub Sponsors "success stories": we bill in a way that's compatible with how companies do business, even if effectively we provide a similar service (which is 95% focused on upstream maintenance, not customer support).
I am not saying it's for everyone, or that Frank should necessarily adopt this model, or that it's the only way (e.g. the Zig foundation raises real amounts of money, too), but I find it frustrating to see over and over again the same conversation:
- "Alice does important maintenance work, she should get professionally funded for it!"
- "How does Alice accept/request funding?"
- "Monthly credit card transactions anchored at $100/mo that are labeled donations"
- no business can move professional amounts of money that way
- "Businesses are so short-sighted, it's a tragedy of the commons!"
Anyone who solicits donations should also sell overpriced books of some sort, because it’s often very easy to get even a $500 book approved as an expense where a $5 “donation” causes hell.
With the year prominently displayed, i.e. "20XX Edition", to reflect when it was current. To help people track how long it has been since they dona-bought their last copy. And so purchase documentation explains repeat purchases.
> However, donating money to an open collective is prohibitively hard for most big companies.
You are absolutely correct. However, that's the mechanism that Frank has made available, and that's what the comment I was replying to was asking, so I was just connecting the dots between the question and answer.
While it might be frustrating to see non-viable options presented as ways to fund critical FOSS, it's even more frustrating to see blame effectively being placed on the maintainer; particularly because, if companies like Apple really wanted to fund this work, I'm pretty sure they could figure something out.
Anyway, looking at the model you propose, it seems like the main difference is that Frank just doesn't explicitly say "you can retain my services"? Is that all that's stopping Apple from contacting him and arranging a contract?
> if companies like Apple really wanted to fund this work, I'm pretty sure they could figure something out.
Having spent the last ~6 years in big tech consistently frustrated by the rigidity of the processes and finding clever ways to navigate (see: wade through the bullshit), this isn’t as easy as you’d hope. The problem is that someone has to spend a non-trivial amount of time advocating internally for something like this (a “non-standard process”) which generally means asking pinging random people across finance, procurement, and legal how to deal with it and 99% of people will just throw up their hands (especially in this case because they don’t understand the importance of it). If things don’t fit a mold in these big companies, they fall into the event horizon and are stretched out to infinity.
Couldn’t you just go up your chain to the VP or whatever and use their backing / negotiating at the VP level to organize? It might not work for random projects but if Apple is using libsodium for security this could presumably be pitched as an investment into their own software supply chain.
Filippo is another maintainer, of extremely similar open source software with entirely the same customer base, offering (important) advice to a peer, so I don't think policing his tone is helpful here.
I know who he is and what he does. I think we probably disagree on whether that makes the comment in better or worse taste.
Otherwise, I agreed with him, and am genuinely curious whether the stopping factor here is maintainers like Frank simply not saying "you can email me to retain my services"
> if companies like Apple really wanted to fund this work, I'm pretty sure they could figure something out
A reminder that companies are not a hive mind.
Many people at Apple surely would love to funnel piles of money to open source. Maybe some of them even work in the Finance or Procurement or Legal departments. But the overwhelming majority of Apple’s procurement flow is not donations, and so it is optimized for the shape of the work it encounters.
I bet there are plenty of people working at Chick-fil-A who wish it was open on Sundays. But it’s not ~“blaming the user” to suggest that as it stands, showing up on Sunday is an ineffective way to get chicken nuggets.
The idea that donations are the only way they could fund this work is what I was talking about. I'm sure Apple has various contractors and other forms of employees.
It's like suggesting that Chic-Fil-A really does want to open on Sunday, but the only thing stopping them is customers not telling them they want it open on Sunday.
Given the increasing obviousness that there's functionally no oversight of NGOs and government funding, perhaps we just need some NGOs and get government grants for these critical services.
If you donate via GitHub Sponsors to https://github.com/jedisct1 from an individual / personal account GitHub won't take a cut (or pays for it from their own purse) for any credit card processing fees.
Maybe you don't know this but Apple has a donation-matching program. If you make donations to non-profits through some special internal mechanism, the company will send a donation of equal value (up to some limit). If I recall correctly the limit is 30K USD per person.
"When you give money to an eligible organization, we’ll match your donations one-for-one, so your $1 has the impact of $2. And if you choose to donate your time, we’ll contribute $25 for every hour you volunteer. Whether you donate time or money, Apple will match your contributions up to $10,000 a year."
Any non-profit, or just charitable non-profits (aka 501(c)(3))? Unfortunately, the US does not consider producing open source software to be charitable activity.
This can't come soon enough. Python is great for CLIs until you build something complex and a simple --help takes seconds. It's not something easily worked around without making your code very ugly.
It kinda looked like this is the future, about at the same time they introduced WSL, released dotNET for Linux and started contributing to the Linux Kernel - all the while making the bank with Azure mostly thanks to running Linux workloads.
But then they deCided it is better to show adds at OS level, rewrite OS UI as a web app, force harware DRM for their new OS version (TPM requirement) as well as automatically capturing content of you screen and feed it to AI.
The Linux kernel and Windows userspace are not very well matched on a fundamental level. I’m not sure we should be looking forward to that, other than for running games and other insular apps.
$10k gets you a Mac Studio with 512GB of RAM, which definitely can run GLM-4.7 with normal, production-grade levels of quantization (in contrast to the extreme quantization that some people talk about).
The point in this thread is that it would likely be too slow due to prompt processing. (M5 Ultra might fix this with the GPU's new neural accelerators.)
> $10k gets you a Mac Studio with 512GB of RAM, which definitely can run GLM-4.7 with normal, production-grade levels of quantization (in contrast to the extreme quantization that some people talk about).
Please do give that a try and report back the prefill and decode speed. Unfortunately, I think again that what I wrote earlier will apply:
> In practice, it'll be incredible slow and you'll quickly regret spending that much money on it
I'd rather place that 10K on a RTX Pro 6000 if I was choosing between them.
No, but the models you will be able to run, will run fast and many of them are Good Enough(tm) for quite a lot of tasks already. I mostly use GPT-OSS-120B and glm-4.5-air currently, both easily fit and run incredibly fast, and the runners haven't even yet been fully optimized for Blackwell so time will tell how fast it can go.
No… that’s not how this works. 96GB sounds impressive on paper, but this model is far, far larger than that.
If you are running a REAP model (eliminating experts), then you are not running GLM-4.7 at that point — you’re running some other model which has poorly defined characteristics. If you are running GLM-4.7, you have to have all of the experts accessible. You don’t get to pick and choose.
If you have enough system RAM, you can offload some layers (not experts) to the GPU and keep the rest in system RAM, but the performance is asymptotically close to CPU-only. If you offload more than a handful of layers, then the GPU is mostly sitting around waiting for work. At which point, are you really running it “on” the RTX Pro 6000?
If you want to use RTX Pro 6000s to run GLM-4.7, then you really need 3 or 4 of them, which is a lot more than $10k.
And I don’t consider running a 1-bit superquant to be a valid thing here either. Much better off running a smaller model at that point. Quantization is often better than a smaller model, but only up to a point which that is beyond.
You don't need a REAP-processed model to offload on a per-expert basis. All MoE models are inherently sparse, so you're only operating on a subset of activated layers when the prompt is being processed. It's more of a PCI bottleneck than a CPU one.
> And I don’t consider running a 1-bit superquant to be a valid thing here either.
Yes, you can offload random experts to the GPU, but it will still be activating experts that are on the CPU, completely tanking performance. It won't suddenly make things fast. One of these GPUs is not enough for this model.
You're better off prioritizing the offload of the KV cache and attention layers to the GPU than trying to offload a specific expert or two, but the performance loss I was talking about earlier still means you're not offloading enough for a 96GB GPU to make things how they need to be. You need multiple, or you need a Mac Studio.
If someone buys one of these $8000 GPUs to run GLM-4.7, they're going to be immensely disappointed. This is my point.
$10k is > 4 years of a $200/mo sub to models which are currently far better, continue to get upgraded frequently, and have improved tremendously in the last year alone.
This almost feels like a retro computing kind of hobby than anything aimed at genuine productivity.
I don't think the calculation is that simple. With your own hardware, there literally is no limits of runtime, or what models you use, or what tooling you use, or availability, all of those things are up to you.
Maybe I'm old school, but I prefer those benefits over some cost/benefit analysis across 4 years which by the time we're 20% through it, everything has changed.
But I also use this hardware for training my own models, not just inference and not just LLMs, I'd agree with you if we were talking about just LLM inference.
Because Apple has not adjusted their pricing yet for the new ram pricing reality. The moment they do, its not going to be a $10k system anymore but in the $15k+...
The amount of wafers going to AI is insane and will influence not just memory prices. Do not forget, the only reason why Apple is currently immunity to this, is because they tend to make long term contracts but the moment those expire ... then will push the costs down consumers.
I think it really depends on how it's used. It's a massive accelerant if it's just helping you stitch stuff together. Or when it helps you get unblocked by quickly finding you the missing api you need.
But when it replaces you struggling through figuring out the mental model of what you're doing, then I think you end up learning at a much more shallow level than you would by doing thing manually.
reply