Re: [gentoo-user] replacement for ftp?

[lee]

"Poison BL." <poisonbl@×××××.com> writes:

> On Sat, Apr 29, 2017 at 3:24 PM, lee <lee@××××××××.de> wrote:
>
>> Mick <michaelkintzios@×××××.com> writes:
>>
>> > On Tuesday 25 Apr 2017 16:45:37 Alan McKinnon wrote:
>> >> On 25/04/2017 16:29, lee wrote:
>> >> > Hi,
>> >> >
>> >> > since the usage of FTP seems to be declining, what is a replacement
>> >> > which is at least as good as FTP?
>> >> >
>> >> > I'm aware that there's webdav, but that's very awkward to use and
>> >> > missing features.
>> >>
>> >> Why not stick with ftp?
>> >> Or, put another way, why do you feel you need to use something else?
>> >>
>> >> There's always dropbox
>> >
>> >
>> > Invariably all web hosting ISPs offer ftp(s) for file upload/download.
>> If you
>> > pay a bit more you should be able to get ssh/scp/sftp too.  Indeed, many
>> ISPs
>> > throw in scp/sftp access as part of their basic package.
>> >
>> > Webdav(s) offers the same basic upload/download functionality, so I am
>> not
>> > sure what you find awkward about it, although I'd rather use lftp
>> instead of
>> > cadaver any day. ;-)
>> >
>> > As Alan mentioned, with JavaScript'ed web pages these days there are many
>> > webapp'ed ISP offerings like Dropbox and friends.
>> >
>> > What is the use case you have in mind?
>>
>> transferring large amounts of data and automatization in processing at
>> least some of it, without involving a 3rd party
>>
>> "Large amounts" can be "small" like 100MB --- or over 50k files in 12GB,
>> or even more.  The mirror feature of lftp is extremely useful for such
>> things.
>>
>> I wouldn't ever want having to mess around with web pages to figure out
>> how to do this.  Ftp is plain and simple.  So you see why I'm explicitly
>> asking for a replacement which is at least as good as ftp.
>>
>>
>> --
>> "Didn't work" is an error.
>>
>>
> Half petabyte datasets aren't really something I'd personally *ever* trust
> ftp with in the first place.

Why not?  (12GB are nowhere close to half a petabyte ...)

> That said, it depends entirely on the network
> you're working with. Are you pushing this data in/out of the network your
> machines live in, or are you working primarily internally? If internal,
> what're the network side capabilities you have? Since you're likely already
> using something on the order of CEPH or Gluster to back the datasets where
> they sit, just working with it all across network from that storage would
> be my first instinct.

The data would come in from suppliers.  There isn't really anything
going on atm but fetching data once a month which can be like 100MB or
12GB or more.  That's because ppl don't use ftp ...

> How often does it need moved in/out of your facility, and is there no way
> to break up the processing into smaller chunks than a 0.6PB mass of files?
> Distribute out the smaller pieces with rsync, scp, or the like, operate on
> them, and pull back in the results, rather than trying to shift around the
> entire set. There's a reason Amazon will send a physical truck to a site to
> import large datasets into glacier... ;)

Amazon has trucks?  Perhaps they do in other countries.  Here, amazon is
just another web shop.  They might have some delivery vans, but I've
never seen one, so I doubt it.  And why would anyone give them their
data?  There's no telling what they would do with it.


-- 
"Didn't work" is an error.