What is your viewpoint of the "The Cloud"?

[ralphbsz]

http should be used for web, not for all whatever file transferts.

Why not? It is practical, available everywhere, well debugged and understood, and very efficient. It works much better than ftp and scp protocols.

yeah, but what about cloud google drive, do you use a tiny file to make it work properly?

Honestly, I don't use Google Drive (or DropBox or similar services) very much. But I do use them for small and big files (sometimes just a few lines of text, to securely transfer them to other people, sometimes for multiple GB as a backup mechanism). As far as I can tell, its performance is purely limited by the bandwidth of my connection, so performance is "trivial", as good as it can be.

Web browser, and resource and memory wasting.

First, you don't have to use a web browser to use cloud in general, cloud storage in particular, and Google Drive/Dropbox as specific examples. You can do it with wget, curl, and http(s) libraries from your favorite programming language (I'm particularly fond of doing it in Python, but tastes on that vary).

Second, you are likely running a web browser on the platform anyway, to read the news, find out what the weather is, and chat on the FreeBSD forum. Asking that web browser to do one more thing (upload and download a file) is very efficient, because it is already being run and in memory. No need to fire up a whole separate process groups, with shells, context switches, shared libraries, and all that gunk.

SCP is far more efficient and free.

I've yet to run a browser that is not free.

And scp is really not efficient at all. Look at the protocol, look at the number of packets that are required, look at how security and dependability are implemented, look at the CPU footprint. It is a bad design, being a hybrid of the very outdated rcp protocol with ssh stuff tacked on. And finally look at the fact that modern production implementations of networks and TCP stacks are optimized for the most common protocol, which is http(s), which is one of the reasons it just runs better than most other protocols.

Im not saying that scp is bad and should never be used. There are good use cases for it, for example involving strong crypto. But making it the default for all file transfer cases is insane.

 

[Spartrekus]

I am a bit surprised by your post, actually, from you. What's going on, ralphbsz?

First, you are likely not to run a browser with a desktop.
That's Unix. This is much more.
It means that you haven't discovered Unix sufficiently. Maybe you forgot. But, it will come back, this very surely. FreeBSD is likely the most attracting OS today. This can even take >20 - 40 years to learn how to take advantage of all power tools, that are available, even how to read the news and to chat with others, in an efficient, user friendly way without a single desktop.

You don't want to write SCP in Python, right? Do you?

http is of course good for reading a webpage (html 3.x or 4.x), but understand that the use of port is of primary importance.
SCP was an example.

Likely let's see that all things have a right place. It is usually good to use what works, without looking for superficial.
Eg. Socks are in the drawer, not lying all around.
Likely, you don't go to mac donalds to eat for both lunch time and in the evening daily.
Same for file transferts.

Feel free to have a look in this direction, you can't find better way to start a small, efficient connection...

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <arpa/inet.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <sys/socket.h>
...

Think UNIX:

 

[Spartrekus]

Which scp, small executable will download all my files from google drive ?

I would like to remove all its content from cloud google drive.

How to do all files of google drive without a browser with FreeBSD under a console, ideally with wget ?

 

[jasonvp]

My opinions on the cloud are as follows; but before I get to them a full disclosure is warranted. I'll be going to work for a company that provides public/private cloud services. Which company isn't material to the discussion, but I felt it necessary to at least get that out there.

With that out of the way: The cloud is called a bunch of things by a bunch of different folks. The most amusing of them is, "It's just someone else's computer." And, to an extent: it is. At its core, you're using time on someone else's server(s) versus your own. This has a lot of benefits depending on the circumstances you and/or your company find yourselves in. For one: you can stay out of the real estate business. That's basically what data centers are these days: real estate. If you don't need your own data centers (for larger companies) or your own racks in someone else's data centers (for smaller companies) you begin to save buckets of money. No longer do you have to manage the physical hardware, the support contracts, the RMAs when they inevitably break. You don't have to manage floor tile space, electrical loads, and cooling. All of that is handled by: someone else.

Second, with "someone else" focusing on the hardware, power, space, networking, etc: your teams can focus on designing and building your system and/or application. There are a few mindset shifts that have to happen when migrating to the cloud and that's one of them: "It's all software, man..." That can be a benefit and a cost at the same time: the folks you have on staff that already "get it" with respect to software will make an easy transition to this paradigm. Those that are more anchored in the hardware side of things are going to need to learn a new set of skills.

Third, with very few exceptions, a cloud provider's infrastructure is going to be vastly larger and far superior to yours. These guys generally build whole new data centers as if it were a "just add water" sort of thing. They have the technical know-how to get these new data centers up and running, complete with full network connectivity to the world, in no time. And with that expanded footprint, they can offer you the ability to build a very robust application or system. YOU have to do the work there, though. It's not automatic in any way. Your application needs to spin up (and spin down!! ($$)) when it's needed, and you need to put the intelligence in it to say, "part of it can live over here on this coast, part of it can live over there on that coast, and another part of it can live way over yonder, in that country." In other words: a distributed app. The cloud provides the expensive part of that: the distributed infrastructure. YOU need to provide the distributed application(s). Things may happen to that infrastructure that are completely out of your control. For instance: a data center could go offline due to a power problem. Or some evil back hoe slices yet another bundle of fiber as they're so wont to do. Your app needs to be written in such a way that it can deal with those issues and continue functioning. That's another of the mindset shifts.

Fourth, as I mentioned earlier, the cloud makes it easier to spin up resources as you need them and then spin them down when you don't. This is yet another mindset shift that has to happen; otherwise your monthly bills will be exceptional. It generally makes little sense to move something to the cloud that needs ALL of its resources running ALL of the time. That will get costly, quickly. A better idea is to have apps that can scale as requests and requirements come in. We need a few more Apache servers here. We need a few more app servers over there. OK, those VMs are done doing their thing; shut them down. Spin up a couple more of there.

Moving to this new world is challenging. Far more so for us network guys, for instance, because we're out of the loop for the most part. All of those years we spent architecting and engineering scalable networks are, for the most part, not needed any longer when it comes to a company moving to the cloud. Different skill sets are required, which is actually OK; it just means we continue to grow. And we're still around should the company decide to do a hybrid or mixed setup. Ultimately, I expect that's what's going to win out in the end.

 

[drhowarddrfine]

jasonvp I completely agree with everything you said--it's what some of us have already said--but you collected it into one nice post. The only thing I would add is that there are some services "cloud" companies offer but you touched on those, too. What bothers some of us is that people are led astray into thinking "the cloud" is somehow different from "a bunch of servers" and, other than services that might be provided by a cloud company, they are one and the same.

What really gets me is reading technical people's writings where they, too, have gotten sucked into the "cloud" marketing term, which, originally, was for lay people to help them understand the internet by using a nebulous term.

 

[PMc]

The cloud provides the expensive part of that: the distributed infrastructure.

The more interesting point seems to be the distributed workforce: since all the skill is now provided by The Cloud, all the jobs that require skill can now be done in India. So the local workforce can be reduced to those highly qualified guys who are able to count money (called "managers"), and some on-site support hired for minimum wages, that way guaranteeing the customer 100% skillfree service, as required by ITIL. Brave New World at it's best.

 

[jasonvp]

The more interesting point seems to be the distributed workforce: since all the skill is now provided by The Cloud, all the jobs that require skill can now be done in India.

I think that's a rather cynical and incorrect way to interpret what's going on. You still need geeks who understand the guts of an operating system as well as the guts of your application. You still need geeks that can think about how to best distribute your digital work load as well as what parts of the work loads get moved to the cloud.

To be perfectly honest, if you're a professional sys admin, this move to the cloud should be relatively easy thing to do assuming your apps are ready for it.

 

[PMc]

I think that's a rather cynical and incorrect way to interpret what's going on.

It's my personal experience, as a customer as well as somebody seeking employment. As a customer I hardly rememeber the time when I got anything else than skillfree service. Nowadays the customers are only filler stuff for the business, counted and averaged by metrics - everything else becoming crap, the people are now the product.

But then, this now is a wonderful world for the zuckerbergs and the scrumbag consultants. But there are the others, those of us who started in 1985 or 1990 to build up the Internet, out of passion, just because it was a damn great thing - while making our living by serving burgers at donald. Those of us who then figured that the companies, trying to go client/server, struggled with exactly the same problems we had already solved (the only difference being that the companies employed an unintellegible obscure language, making things appear needlessly complicated, and smoke-screening by buzz-words that what would really count: reliability and proper engineering).
Those of us are now superfluous, and usually too old to go back to the burger serving business.

You still need geeks who understand the guts of an operating system as well as the guts of your application. You still need geeks that can think about how to best distribute your digital work load as well as what parts of the work loads get moved to the cloud.

In India, maybe.

To be perfectly honest, if you're a professional sys admin, this move to the cloud should be relatively easy thing to do assuming your apps are ready for it.

I have nothing to do with that, I'm just an ordinary guy.

 

[Deleted member 30996]

I keep my head in the clouds.

My data stays on redundant USB sticks with passwords encrypted by security/bcrypt. That works for me. I'm not a movie star and don't have any nude pix or I would use the cloud.

 

[Spartrekus]

I keep my head in the clouds.

My data stays on redundant USB sticks with passwords encrypted by security/bcrypt. That works for me. I'm not a movie star and don't have any nude pix or I would use the cloud.

There is a way that you can replace your USB.

There is www and ssh for replacing your clouds on microsoft and google. Know that those two use all your data on the clouds and make huge profits *dollars*.

So here is the good trick for your FreeBSD Data Solution.
Caution those commands are Free, Libre, quick, portable and working.
Bonus: It works on any today Unix installations.
Extra Bonus: it is fast, and it is recommended to protect environment, for a sustainable solution.

what about fun tunnel to get so anywhere?

    echo easy sshfs tunnel for the newbie
    ssh -p 6222  username@distantsshopenport  -L   3000:192.168.1.2:22
    cd ; mkdir drive ; sshfs -p3000 username@localhost:/  drive

PC anywhere -> wwww -> distant gate pc with router, open port on port 6222 -> 192.168.1.2 on port 22 this is your harddisk on freebsd server - libre - free and with ZFS or UFS or Raid.
3000 is like a virtual one, used to connect.