Depends what level of accuracy you want. I just started in a computational chemistry lab so I'll probably get some details wrong, but for small systems, you can use a method called CCSD(T) for up to ~20 atoms, but it scales O(N^7). I've been mainly using DFT for the systems I've been simulating, which scales O(N^3). I've been running a system with about 50 atoms with a decent basis set (how the orbitals are modelled), and it takes about 30 minutes for each optimization step with 24 cores and 48 GB of RAM.
DFT works in many cases, but in some cases it doesn't estimate the energy right, due to how it bypasses some correlation calculations. Bonds are extremely sensitive to energy calculations, so you need to get super close to the actual energy in order to get useful results.
Anyways, someone with more experience here could probably add more, but that's what I've picked up so far.
Cool details, thanks. To help me understand your life, what would be like a one year and a five year research goal for you? I never spent time in lab sciences so it’s kind of a black box for me.
Disclaimer: I'm only a freshman, so there's still a ton I don't know :)
Right now the lab is having me get comfortable using software like Gaussian and ORCA by simulating a bifurcating reaction. This is a reaction that, depending on the catalyst's momentum, will change what site it bonds to (it makes either a 6-membered or 7-membered ring). I'm finding the intermediate states (where the molecule is most stable) and transition states (the tipping point), and then running trajectories to see which output is more likely.
Once I've finished simulating that, I should be comfortable enough with the process to jump on the bigger project, which is machine learning interatomic potential (MLIP) model distillation. There's a lot of exciting work around speeding up DFT methods by using machine learning (note this is not generative AI, it's merely predicting the molecule energy based on atomic positions). So my one year goal is to get on that project and start contributing.
My five year goal is to, well, graduate. But then I'll probably do a PhD in computational chemistry, since I'm really interested in ways to speed up and scale existing methods. My big dream is to simulate large biological systems while still having bond formation and breaking, to automatically elucidate biochemical pathways, but there's still a lot of steps in-between.
I hope and pray that your research helps to make the world a better place and that the rest of us can use your knowledge to help to make the world a place which merits your research.
Thank you for the kind words! I've been wanting to do this research precisely because of firsthand experience with how hard chronic illness can be, and I'm hoping to attack it with a systems approach.
I haven't seen that website before, but it sounds pretty accurate from what I've heard. It's insane how high of a mountain needs to be climbed just to catch up to the state-of-the-art, and how much work is needed to push through to figure out something truly new.
I view the difficulty/breadth and depth of knowledge necessary to begin a PhD as a tribute to humanity's successes (and hope to embark upon my own after retirement, though it'll be in CNC machining).
do you think quantum computers would help simulating this? I've seen contradictory opinions from the experts - it can in theory but not really in practice (even assuming sufficiently large quantum computers will be built)
yea im doing my masters in dft research so ik abt this.
depends what u want 2 simulate! chemists more do molecular dynamics type stuff and will use experimental data for fitting data etc. like uh what surface of a metal water will react with from thermodynamics or something. (that isnt my field lol i just know a lot of catalysis guys.)
truly ab initio methods involve figuring out electronic properties from scratch like ionization energy or bandstructure. the real issue is that we dont have exact relations for the exchange and correlation terms. we can know the kinetic energy and charge screening, but we dont know how the electrons are interacting with each other. generally the xc term is treated as a function of electron density or its gradient (see: lda, gga, meta-gga) but there are so many different ways to approximate that. different models are good for different applications also, like transition metals vs organics. and then theres the issue of basis sets (most people use gaussian basis sets that have been tuned over many years but theres also plane waves and finite element methods) which can also change results. and even once u have a decent approximation of density you can try perturbative methods (GW family, delta scf i count also) to try and improve the approximation.
i am rambling and typing this on my phone. essentially yes, but often calculations are a little inaccurate. but more accuracy has a higher computational cost, which makes it hard to run larger simulations. tradeoffs of engineering. hope this was coherent.
If you want to get pedantic we can't simulate anything with complete accuracy in the absence of a theory that encompasses all the known forces. Which we don't have. (Damn you gravity. Can't you just get along with the others)
To a useful level of accuracy we can certainly simulate water. And we can do the same for a single proton for some definitions of useful (but not other definitions).
Am I right with my assumption that by "fundamentally different problem", you mean we lack a good simulation model, but that the number of degrees of freedom would actually be manageable?
To simulate a proton you need to solve a strongly coupled highly relativistic SU(3) gauge theory (naturally non-abelian i.e. the force carrier field itself carries charge and is self-interacting at tree order) problem with constituents that have masses orders of magnitude below the relevant energy scales (i.e. you have many matter AND force particles that can pop in and out of existence and they all strongly interact with one another).
To simulate a water molecule you do so with a weakly coupled SU(1) gauge theory (light does not interact with itself at tree order) problem where the masses of all constituents are orders of magnitude above the relevant energy scales (you can think of it as the electrons and nuclei and particles coming in and out of existence are contained in a renormalization scheme).
We have "good simulation models" of both, but the former is extraordinarily complicated compared to the latter for the reasons stated above.
I'm working on an app similar to Anki, where I can make my own cards for spaced repetition, organize them by folders and tags, and can run different queries to get groups to practice from. I'm curious what features people wish Anki had. I didn't love it when I tried it.
I want to use my app mainly for language learning, but as a demo, I also have some geography cards that zoom in on a country on the world map, for the front side.
Interesting. My projects would benefit from those more convenient clones into closure captures. The other point where Rc is uglier than, say, Swift, is the explicit `.borrow()` and `.borrow_mut()` everywhere. I wonder if that could also be made more convenient/high-level without sacrificing the control over high performance (like C++) that got me to use Rust in the first place.
Some Clojure fans once told me they thought datalog was better than SQL and it was a shame that the relational DBs all used SQL. I never dug into it enough to find out why they thought that way.
Basically Datalog is much less verbose than SQL, imposes much lighter tariffs on factoring out views, and supports transitive closure enormously better. I started http://canonical.org/~kragen/binary-relations off with a simple nonrecursive query for which the SQL translation (given below) is already criminal and whose properly factored SQL solution merits the death penalty.
Recent additions to ANSI SQL have added the capacity for recursion, so it's no longer completely impossible. But they have three big disadvantages:
1. They accidentally made SQL Turing-complete. Datalog queries, by contrast, are guaranteed to terminate.
2. They're still extremely clumsy to use.
3. Because of #1, they're often not implemented fully, so they are hard to rely on.
Yes, #1 basically means that they screwed up the design from the get go, since it is impossible to reap the actual benefits of Datalog when the language you evaluate is not, in fact, Datalog. Recursive queries have the ability to perform arbitrary computation in projections, so for starters any top-down evaluation strategy or hybrid evaluation such as magic sets is ruled out.
At first blush that sounds impossible. Maybe there's a clever solution that would occur to someone if they spent a few months working on it, or maybe not.
I struggle to understand the Clojure/Datomic dialect, but I agree generally. I recommend Percival for playing around with Datalog in a friendly notebook environment online: https://percival.ink/
Although there’s no “ANSI SQL” equivalent standard across Datalog implementations, once you get a hang of the core idea it’s not too hard to understand another Datalog.
I started a Percival fork that compiles the Datalog to SQLite, if you want to check out how the two can express the same thing: https://percival.jake.tl/ (unfinished when it comes to aggregates and more advanced joins but the basic forms work okay). Logica is a much more serious / complete Datalog->SQL compiler written by a Google researcher that compiles to BigTable, DuckDB, and a few other SQL dialects (https://logica.dev/).
One area Datalog is an order of magnitude easier is when working with recursive queries / rules; this is possible in SQL but feels a bit like drinking playdough through a straw. Frank’s Materialize.com has a “WITH MUTUALLY RECURSIVE” SQL form (https://materialize.com/blog/recursion-in-materialize/) that’s much nicer than the ancient ANSI SQL recursive approach, we’re evaluating it for page load queries & data sync at Notion.
Feldera has a similar form for recursive views as well (https://www.feldera.com/blog/recursive-sql-queries-in-felder...). I like that Feldera lets you make each “rule” or subview its own statement rather than needing to pack everything into a single huge statement. Main downside I found when testing Feldera is that their SQL dialect has a bunch of limitations inherited from Apache Calcite, the Materialize SQL dialect tries very hard to be PostgresSQL compatible.
> Main downside I found when testing Feldera is that their SQL dialect has a bunch of limitations inherited from Apache Calcite
At Feldera, we're adding features to our SQL over time, by contributing them upstream to Calcite, making it better for everyone. Mihai Budiu, who is the author of the Feldera SQL compiler, is a Calcite committer.
Thanks for contributing. I see Mihai implemented the UUID type in Calcite (https://issues.apache.org/jira/browse/CALCITE-6738) back in January which is one of the issues I hit, so for sure my experience with Feldera is 6 months out of date and y'all move pretty quick.
Most of what I mean is places where Feldera/Calcite has slightly different syntax from Postgres for things. For example, Postgres syntax for cast to bigint is `some_expresion::bigint` although Postgres also supports ANSI SQL `CAST(some_expression AS bigint)`, most examples I find in the wild and in my own Postgres SQL use the Postgres special syntax. JSON syntax also differs; Feldera uses its own pretty elegant `VARIANT` type and `some_expression[key_expression]` to access properties, where Postgres calls this `json` or `jsonb`, and uses `some_expression->key_expression` to access properties. In those cases it's not like Feldera is wrong or lacks some support, but it's a bit harder to work with for me because I'm so used to Postgres syntax and I need to do some regex replace whenever I bring a query from Postgres over to Feldera.
Definitely not a deal-breaker, I am a Feldera enjoyer, but it does add some friction.
Thanks for the kind words. :) We hear you on the dialect differences.
An interesting case of a user dealing with this problem: they use LLMs to mass migrate SparkSQL code over to Feldera (it's often json-related constructs as you also ran into). They then verify that both their original warehouse and Feldera compute the same results for the same inputs to ensure correctness.
Been trying Rust for some months now. The IDE experience is so much nicer I'm probably sticking with it, but the restrictions on how I write code are still grating on me. I need to learn the language better, but now I still feel like I wanted something more like C++ 2.0 without as much paradigm shift.
I just used Skype the other day. I still find it useful for certain things. Is it that these companies can't maintain a piece of software unless they see a way for it to grow and dominate the world?
Yup. They could have security by allowing the user to see and control the sandboxing that the operating system imposes on the apps. The app store is not required, but they act like it is.
