1 |
"Poison BL." <poisonbl@×××××.com> writes: |
2 |
|
3 |
> On Sat, Apr 29, 2017 at 3:24 PM, lee <lee@××××××××.de> wrote: |
4 |
> |
5 |
>> Mick <michaelkintzios@×××××.com> writes: |
6 |
>> |
7 |
>> > On Tuesday 25 Apr 2017 16:45:37 Alan McKinnon wrote: |
8 |
>> >> On 25/04/2017 16:29, lee wrote: |
9 |
>> >> > Hi, |
10 |
>> >> > |
11 |
>> >> > since the usage of FTP seems to be declining, what is a replacement |
12 |
>> >> > which is at least as good as FTP? |
13 |
>> >> > |
14 |
>> >> > I'm aware that there's webdav, but that's very awkward to use and |
15 |
>> >> > missing features. |
16 |
>> >> |
17 |
>> >> Why not stick with ftp? |
18 |
>> >> Or, put another way, why do you feel you need to use something else? |
19 |
>> >> |
20 |
>> >> There's always dropbox |
21 |
>> > |
22 |
>> > |
23 |
>> > Invariably all web hosting ISPs offer ftp(s) for file upload/download. |
24 |
>> If you |
25 |
>> > pay a bit more you should be able to get ssh/scp/sftp too. Indeed, many |
26 |
>> ISPs |
27 |
>> > throw in scp/sftp access as part of their basic package. |
28 |
>> > |
29 |
>> > Webdav(s) offers the same basic upload/download functionality, so I am |
30 |
>> not |
31 |
>> > sure what you find awkward about it, although I'd rather use lftp |
32 |
>> instead of |
33 |
>> > cadaver any day. ;-) |
34 |
>> > |
35 |
>> > As Alan mentioned, with JavaScript'ed web pages these days there are many |
36 |
>> > webapp'ed ISP offerings like Dropbox and friends. |
37 |
>> > |
38 |
>> > What is the use case you have in mind? |
39 |
>> |
40 |
>> transferring large amounts of data and automatization in processing at |
41 |
>> least some of it, without involving a 3rd party |
42 |
>> |
43 |
>> "Large amounts" can be "small" like 100MB --- or over 50k files in 12GB, |
44 |
>> or even more. The mirror feature of lftp is extremely useful for such |
45 |
>> things. |
46 |
>> |
47 |
>> I wouldn't ever want having to mess around with web pages to figure out |
48 |
>> how to do this. Ftp is plain and simple. So you see why I'm explicitly |
49 |
>> asking for a replacement which is at least as good as ftp. |
50 |
>> |
51 |
>> |
52 |
>> -- |
53 |
>> "Didn't work" is an error. |
54 |
>> |
55 |
>> |
56 |
> Half petabyte datasets aren't really something I'd personally *ever* trust |
57 |
> ftp with in the first place. |
58 |
|
59 |
Why not? (12GB are nowhere close to half a petabyte ...) |
60 |
|
61 |
> That said, it depends entirely on the network |
62 |
> you're working with. Are you pushing this data in/out of the network your |
63 |
> machines live in, or are you working primarily internally? If internal, |
64 |
> what're the network side capabilities you have? Since you're likely already |
65 |
> using something on the order of CEPH or Gluster to back the datasets where |
66 |
> they sit, just working with it all across network from that storage would |
67 |
> be my first instinct. |
68 |
|
69 |
The data would come in from suppliers. There isn't really anything |
70 |
going on atm but fetching data once a month which can be like 100MB or |
71 |
12GB or more. That's because ppl don't use ftp ... |
72 |
|
73 |
> How often does it need moved in/out of your facility, and is there no way |
74 |
> to break up the processing into smaller chunks than a 0.6PB mass of files? |
75 |
> Distribute out the smaller pieces with rsync, scp, or the like, operate on |
76 |
> them, and pull back in the results, rather than trying to shift around the |
77 |
> entire set. There's a reason Amazon will send a physical truck to a site to |
78 |
> import large datasets into glacier... ;) |
79 |
|
80 |
Amazon has trucks? Perhaps they do in other countries. Here, amazon is |
81 |
just another web shop. They might have some delivery vans, but I've |
82 |
never seen one, so I doubt it. And why would anyone give them their |
83 |
data? There's no telling what they would do with it. |
84 |
|
85 |
|
86 |
-- |
87 |
"Didn't work" is an error. |