How do you view a webpage without a browser in a console?

How do you view a webpage without a browser in a console?


  • Total voters
    11

hardworkingnewbie

Well-Known Member

Reaction score: 287
Messages: 277

What's the purpose of that question? First of all there are enough browsers working in console, like Lynx, W3M and others. Lynx also has --dump.

Second you could just use your programming language of choice with an HTML parsing library of your choice, request the page, parse it and save its context as plain text.

Either way is no rocket science, and there are enough premade scripts around for that.
 

Crivens

Moderator
Staff member
Moderator

Reaction score: 1,746
Messages: 2,588

How to you fly to the next continent without using a plane?
A really strong catapult may get you from europe to africa. Other than that, falling down the right mountain in the urals may land you in asia, or tripping on the right border crossing in central america may send you airborn to the next continent. See? Easy. And yes, that last clown I ate was over its best-before date.
 

covacat

Daemon

Reaction score: 535
Messages: 1,086

Code:
$fetch -o - https://www.freebsd.org|xmllint --xpath '//*/text()' --html - 2>/dev/null| grep .|more
 

Alain De Vos

Son of Beastie

Reaction score: 870
Messages: 2,826

This trick does not work on random sites like,
Code:
https://www.theguardian.com/us-news
I think many pages are "too complex"
 

ralphbsz

Son of Beastie

Reaction score: 2,517
Messages: 3,378

Used to use wget, changed to curl. But as Alain said, in practice modern web pages are no longer human-readable and don't consist of a single HTTP download.
 
OP
V

vigole

Daemon

Reaction score: 1,601
Messages: 1,413

Either way is no rocket science
As the title of the poll, i.e. "without a browser".

e.g. you need few hints from a handbook/faq/howto, enough to set some options right, in order to config a intro(4)/man(4) device correctly.
 

richardtoohey2

Aspiring Daemon

Reaction score: 347
Messages: 674

I'm with covacat - just use fetch - it's built-in, nothing to install. That's usually enough to see what's in the page.
 

hardworkingnewbie

Well-Known Member

Reaction score: 287
Messages: 277

As the title of the poll, i.e. "without a browser".

e.g. you need few hints from a handbook/faq/howto, enough to set some options right, in order to config a intro(4)/man(4) device correctly.
If you would have read my whole post - which you did not - you would have also known that you could just use your language of choice with pre-bundled HTML parser.
 

ralphbsz

Son of Beastie

Reaction score: 2,517
Messages: 3,378

But how do you display the pages using those? I'm not talking "with additional tools" like the fetch example earlier.
Emacs, more, ... but for modern complex web pages, that's impractical. For simple stuff, it works great.
Often I actually use it to download pdf files or images.
 

gpw928

Aspiring Daemon

Reaction score: 298
Messages: 643

Currently, I don't (but have in the past):
Code:
[strand.312] $ grep "text/html;" ~/.mailcap
#text/html; w3m -I %{charset} -T text/html; copiousoutput;
#text/html;lynx -dump %s; nametemplate=%s.html; copiousoutput
#text/html;firefox %s; nametemplate=%s.html
#text/html;elinks -dump %s; nametemplate=%s.html; copiousoutput
#@#Sun Jan 19 14:57:59 AEDT 2020#text/html;chrome %s; nametemplate=%s.html
#@#text/html;iridium %s; nametemplate=%s.html
#@#text/html;w3m -cols 72 -I %{charset} -T text/html -s | sed -e 's:^[[:blank:]]*$::' | cat -s | less; copiousoutput
#@@text/html; lynx -dump -force_html -stdin | sed -e 's:^[[:blank:]]*$::' | less -s
#@#text/html;luakit %s >/dev/null 2>&1; nametemplate=%s.html
#@#text/html;midori %s >/dev/null 2>&1; nametemplate=%s.html
text/html;firefox %s >/dev/null 2>&1; nametemplate=%s.html
 

Alain De Vos

Son of Beastie

Reaction score: 870
Messages: 2,826

When i as a European look at U.S. pages i first need to agree on the applicable law, before i can even see the first page. This is interactive ...
Not all internet pages are as simple as freshports .
 

Zirias

Son of Beastie

Reaction score: 1,783
Messages: 3,030

No, that only fetches the markup but doesn't display the page. That's also not on the list.
The markup is just a representation of the document, so sure that's a display. As curl and telnet are on the list, I'm pretty sure that's fine.
 

drhowarddrfine

Son of Beastie

Reaction score: 2,506
Messages: 4,453

Zirias No. He asks "How do you view a webpage..." Looking at or downloading the markup using curl or telnet is not viewing a web page. I take that to mean NOT just wanting to look at the source markup.
 
Top