The ads often crash the app on my tablet and occasionally my phone too. I've taken to refreshing the content and then switching to airplane mode to cut off network connectivity. You don't get images but you can read all the text as articles are downloaded to your device.

"Silicon Valley, disrupt interviews, please!"

They did and that's the problem ;)

Here is the only way to shop Lenovo:

http://psref.lenovo.com/

Every SKU, every option, every detail, in a PDF.

I'd say there's a better way if you're on a budget:

http://outlet.lenovo.com/outlet_us/

http://dell.com/outlet/

http://www.apple.com/shop/browse/home/specialdeals

etc.

In scrolling around, it looks like debt load corresponds to the locations of colleges. Seems simple that if you want to know where there is student debt, you look for students. The delinquency and income correlation is more interesting.

You're rolling along thinking this is just some bad web developer somewhere until you get to the last Disallow...

Could that be a honeypot to trap bots that don't respect robots.txt?

Looks more like a mop-up after a website infection incident.

I have had to do that before. The URLs in that case seemed to have a more random distribution than most of these.

maybe, but why would THAT be the honey pot? it could be any phrase, really.

Well, you'd want them to try to crawl it. Perhaps there was a bot crawling their site for such links and they wanted to identify specifically that one. It seems most likely that such URL's never existed, but this would be a decent way to identify a bot that's otherwise not distinguishable.

Well, I'm on a list now..

You have something against petite young ladies of a modest stature? Sizeist!

This other retailer's page is about the same except for that one entry: http://www.neimanmarcus.com/robots.txt

I don't get it: is that disallow targeting some sort of porn spambot crawler?

They probably made a product with that slug by accident and are doing that to try to get it removed from search engine results.

It makes me wonder if Nordstrom ever stocked the now-defunct Pornstar clothing brand.

Disallow is used to exclude certain URLs from being crawled, in this case it looks like a search query as the store's search uses this pattern: http://shop.nordstrom.com/sr/query .

Maybe some mal-intended inbound links that they don't want to be associated with?

Is this some sort of new terrible negative seo? Associate a target url for a bunch of negative things?

Maybe that was their intention, but that's not how it's done.