Hacker Newsnew | past | comments | ask | show | jobs | submit | phillmv's commentslogin

what if i were purely interested in archiving local community posts? does anyone have experience on scraping without getting banned?

i have no interest in juicing engagement, purely from a "20 years from now it'll be cool to know this existed at all" aspect.


I would guess that low-volume scraping is probably much less likely to cause problems.

i feel like ppl have magical beliefs about type systems. just because it's _probably_ (did it use unsafe?) memory-safe doesn't mean it does what you want it to do

That's obviously true. On the other hand its also true this is more likely to work because it is rust compared to python or js for example. And that's not only because of memory safety. It's because static typing gives an automatic proof of a certain level of correctness of the code. That correctness is correlated with correct business logic bugs. So it is valid argument to make.

Of course that doesn't mean that there are no businesses logic bugs.


javascript is like… unusually messy and weird, so maybe that colours most people's perspective. you don't have to worry about type coercion and weird kinds of equality and so on in python and ruby to anywhere near the same extent.

>Type systems restrict which programs can be expressed and increasing expressiveness often requires increasing type-system complexity (which, speaking from experience, both humans and agents will struggle with). Plus they are not the only mechanism to assert correctness (they only validate a subset of your program correctness and do not replace tests)

This articulates a lot of my own thinking wrt type systems, speaking as a downstream user without a lot of exposure to prog language theory, and I wish this debate were more often framed in these terms.

Another reply to this comment hinted that it might be more about giving LLMs feedback loops and that to me also seems like a more likely mechanism.

I'm not an elixir user but I've watched it from a distance over the years – thank you for your efforts and your experimentation.


i like how when a company obviously discriminates against women and minorities by hiring almost entirely white guys that's fine that's to be expected but if you try to fix that discrimination it's an evil conspiracy


The fact that the company is majority white does not make discrimination legal. If the Perkins Coie wants to do things like anonymize its interviews, or send fake interview packets to its recruiters and looking for disparities in call back rates then that would be a genuine attempt at identifying potential discrimination.


the discrimination already happened! it's not possible to end up 80% white guy without discriminating. it's curious that the status quo isn't nearly as concerning.


> it's not possible to end up 80% white guy without discriminating?

This is untrue, though. The fact that a company does not have representation that is exactly equitable with the general population is not evidence of discrimination.

In fact, you can end up with disparities much larger without discrimination. It's even possible to actively discriminate against a group, and still have that disadvantaged group be overrepresented by a factor of 3 or 4.

That was the case with the Harvard admissions lawsuit. Even though the university was actively discriminating against Asian applicants, the undergrad population was ~20% Asian, despite ~6% of the applicants being Asian.


>The fact that a company does not have representation that is exactly equitable with the general population

i didn't say exactly equitable, i said 80%. it's not possible to have 80% white guys and not be discriminatory.

you're making a bad faith apples to oranges comparison, to say nothing of the merit of Students for Fair Admissions v. Harvard. your viewpoint and disinterest is very clear, i don't know why you even bother arguing about it.


Yes, it is possible to have an 80% white organization without discrimination. Perkins Coie is ~80% white but is gender representation is much closer to parity [1], so I'm not sure why you're referring specifically to "white guys".

The relevance of SFFA vs. Harvard is to demonstrate that it's possible to have a substantial overrepresentation - over 3x in the case of Asians at Harvard - despite actively discriminating against the overrepresented group. Whites are only ~1.2x more common at Perkins Coie relative to the general population.

You can keep repeating the line that because a company has X% of Y race it must be evidence of discrimination as many times as you want, repetition doesn't make it true.

1. https://perkinscoie.com/about-us


not going to debate bad faith right wing cause celebres arguing the inverse.

i see it with my own two eyes when i have to yell at recruiters to stop bringing me all-male candidate slates, i see it from studies in social sciences, and i know it from having seen the progression over my own career and listening to my female colleagues. at the end of the day a lot of men don't see women and minorities as people with full agency.


If you did a cursory review of the social sciences on this topic, you'd know that the demographics of people going through the prerequisites to working at a place like Perkins Coie does not match the general population. College attendance has racial disparities, as does law school attendance. Whites have half and 1/3rd the fail rate at the bar examination as compared to Latin and Black people respectively. Even absent discrimination, there are plenty of factors that drive law firms to have larger representation of white people.


> it's not possible to end up 80% white guy without discriminating.

That is not remotely true. Individual choices, as well as experiences which shape the candidate pool, can cause such lopsided numbers. In my view, the single biggest problem with the (quite well intentioned) diversity initiatives is that they assume, without evidence, that any organization with lopsided demographics must therefore be discriminating. But that is a fallacy and undermines the entire endeavor they are engaged in.


there's loads of evidence, y'all just don't like engaging with it cos you don't like the answer because fixing it is work and fundamentally women and minorities are not seen as people with full agency.

i'm old enough to remember when software engineering conferences were _2%_ female. it's exhausting to be having this same conversation decades later.


What is the evidence? Did we send identical applications, differing only by ethnicity, to Perkins Coie, and did they respond to the non-white applicants at lower rates? Did Perkins Coie institute policies like withholding bonuses if leaders hired too few white applicants? What indicates that Perkins Coie is preferring white applicants over equally qualified Black or Latin applicants?

You insist there's evidence of discrimination, but all you've done is point to the % of white people at the company and insist it's too high.

But as a counterpoint, 40% of the developers at my company are Asian, despite them making up 6% of the US population. That's an overrepresentation of over 6X. In fact, whites are slightly underrepresented. Does that mean we're discriminating against non-asians? Is this evidence that whites are discriminated again, on account of their underrepresentation? Of course not.


This is absurd. Why is it possible to end up with a majority black, tall, male, NBA team without pre-selecting on basis of race, height, or sex?


>Are Go backend building custom auth, admin, DB ORM/migrations/auto migrations, templates, email, dev server etc for each project?

lmao, basically, yes. except when you bring this up ppl think it's not a big deal / a means for self-expression. having to sort through which libraries you prefer to glue together is a kind of freedom, if you squint hard enough.


I get the "you're reinventing the wheel!" thing, but trying to fit a formula 1 car wheel to my horse-and-cart system really isn't working for me. I need a different wheel, so I'll just build my own. Luckily there's a ton of good libraries available that have all the tricky bits covered so it's relatively easy to build my own wheel.


>You can advocate for sustainability … while being strongly capitalist just fine. […] excluding a large intersection of people otherwise interested in the non-political parts

the far-right is literally trying to make it illegal for companies to say they're taking environmental concerns seriously (i.e. ESG bans in tx, fl, etc). in 2026, sustainability is not apolitical.

(it's _never_ been apolitical but i will spare you that lecture.)


Most of the far-right are idiot contrarians (my personal view) and they have no claim on capitalism.

Just because far-righters are against sustainability does not mean you have to be a post-marxist anarchist (or w/e) just to be for it.


> does not mean you have to be a post-marxist anarchist

if you oppose a far-right project, what are they going to call you? lol. here is your membership card for the cultural marxist party, welcome aboard comrade!

i just think it's folly for people to complain that "not destroying the environment" is a partisan issue in 2026. being a nice person is a partisan issue in 2026.


Despite what "far-right" groups may claim, politics isn't one giant us-versus-them war; I refuse to stoop down to their level.

The US right likes to call their opponents pedophiles, but it would be ridiculous for anyone to adopt that label for themselves because of it.


> if you oppose a far-right project, what are they going to call you?

In Germany literally "linksgrünversifft", which loosely translates to filthy green/left advocate.

Which is, critically, completely unrelated to both communism and anarchism; while a lot of the political spectrum opposes the far right, only a tiny fraction are actual communists or anarchists (at least in Europe).

I do feel your pain that a significant part of modern politics involves completely indefensible, irrational and irresponsible positions, but that still does not mean you have to be marxist/anarchist or even anti-capitalist just to oppose that idiocy.


it helps that there's a regulatory agency that verifies the cleanup happened! if the 4th of july might get canceled the following year ppl might be more aggressive around cleaning up.


Participants also have to feel like they are part of the event rather than passive spectators.


Do they? Japanese clean up after themselves when they fill the parks at cherry blossom season or a fireworks festival. Seems like you just need to feel responsible for yourself


Could be because people growing up in Japan are taught that they're an intrinsic part of any place, event, or group of people that includes their presence. Kids in classrooms in Japan are helping clean up together with everyone else at age 4.

It's kinda the opposite of "responsible for yourself," it's a civic sense that extends to include everyone and everything around you - including things that weren't directly caused by you-as-individal.

In the case of the cherry blossoms, they were planted for the enjoyment of the people, and thus the people who come to enjoy them are a part of that system. The cherry blossom viewing events where thousands of people come to picnic, only is a "thing" because thousands of people come - everyone there is a participant by virtue of attending. Thus they hold part of the responsibility for the outcome of the event and the aftermath.


Most people clean up after themselves, even in hyper individualist America.

The Japanese clean up after everyone including the shitty few who don't clean up after themselves.


no my experience. Go see a public park in Japan on weekend at the beginning of April. Every portion of the park will be covered in people. At the end of the day they'll all go home and the park will be mostly clean except for the garbage collection area.

Compare that to USA, Go to the National Mall (the grass in DC) on the 4th of July. See it's covered in people. Check on the 5th of July. The entire place will be covered in trash and refuse.


That isn’t in contradiction to what I said. It is further evidence of my argument.

If 2% of the people on the mall on the 4th don’t clean up after themselves and the 98% don’t pick up the slack, it will be disgusting, even if 98% of the people are responsible for their own trash. Americans draw a hard line on trash that is theirs and trash that isn’t. Japanese do not.

My theory is that in Japan people see it as their responsibility to clean up, regardless of who made the mess. In the west, most people are only concerned with their own mess.


i have wanted to be able to compile ruby to a binary for some time – and have dreamed of poking at this problem with claude – so this is pretty cool.

if you can get a Rack compatible web server to build… i'd waste some serious time playing with this.


giving these things control over your actual computer is a nightmare waiting to happen – i think its irresponsible to encourage it. there ought to be a good real sandbox sitting between this thing and your data.


Hard agree. I'm on vacation in Mexico atm and when I get back I get to repair my OS because I gave codex full control over my system before I left. Was rushing trying to reorganize my project files to get up to the GitHub before I left. Instead it deleted my OS user profile and bonked my system.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: