Хвост файла по FTP

4017
jwa

Я пытаюсь получить доступ к большим файлам журнала на удаленном сервере с моего рабочего стола Windows. У меня есть только FTP-доступ к этой удаленной машине, но не SSH-доступ.

На данный момент я использую WinSCP, чтобы вытянуть весь файл через FTP. Это означает, что я должен передавать полный файл каждый раз. Однако, учитывая, что это лог-файл, мне, вероятно, понадобятся только последние несколько строк.

Это особенно расстраивает, так как моя пропускная способность сильно ограничена, поэтому передача всего файла занимает несколько минут.

Если бы у меня был доступ к оболочке, этого можно было бы легко достичь, используя что-то вроде tail -100получения последних 100 строк.

Я хотел бы найти решение для выполнения этого через FTP. Обратите внимание, что это не обязательно должен быть сплошной хвост, достаточно одного раза.

11

3 ответа на вопрос

8
Martin Prikryl

What you want to do, can definitely be done with FTP. Technically it's the same, what any FTP client does, when resuming an interrupted file download.

Though from a user perspective, I do not know, if any FTP client supports an explicit download of only given number of trailing bytes.

But some FTP clients will definitely allow you to download new trailing contents of file that you have previously downloaded.

Particularly with WinSCP, just initiate a log file download. Then, on the Overwrite confirmation prompt, select the Resume (it's in drop down menu of the No button). Note that the option is obviously available, only if the source file is larger than the destination file.

If you really want to download only the last few lines of the log, you can cheat WinSCP, by creating a dummy local file with a size bit smaller than the log file before you initiate download.

You can also easily automate the above trick:

fsutil file createnew mylog.log 100000000 winscp.com /command "open mysession" "get -resume /path_to_log/mylog.log" "exit" 

For alternatives to the fsutil, see Quickly create large file on a windows system?

With a more effort you can modify the script to first check log file size and automatically calculate dummy file size few kilobytes smaller than the log size.

0
ChrisInEdmonton

I don't know how you can do this using WinSCP, and indeed it may not be possible, but in general, you should be able to use the REST command. See RFC 3659. It is not clear to me whether you can start downloading from late in the file, though, or whether you need to already have received the markers. RFC 3659 indicates this is possible, at least in STREAM mode, though:

STREAM mode transfers with FILE STRUcture may be restarted even though no restart marker has been transferred in addition to the data itself. This is done by using the SIZE command, if needed, in combination with the RESTART (REST) command, and one of the standard file transfer commands.

0
Znik

I didn't hear about any software that supports your needs. But if you are programmer, shou should understand how to write this in Perl.

Here i found example how to get all file. This isn't your resolve, but it help you understand how to do: http://www.perlmonks.org/?node_id=907019

another, help for library Net::FTP : http://search.cpan.org/dist/libnet/Net/FTP.pm

using this you can open ftp connection to server (first example) make authorization (first example)

second URL help you how to get last part of file using methods: ->ascii set transfer mode to ascii because this is logfile ->size ( FILE ) get size of file for counting where you should start ->restart ( WHERE ) set file coursor where from in the file you want start reading ->read ( BUFFER, SIZE [, TIMEOUT ] ) directly read small SIZE part into BUFFER variable

another functions: length(BUFFER) tell you how much data you get. If it's ZERO, you propably reached EOF print ("$BUFFER\n") simple print buffer content to the screen

Notice, this is not completly resolution but a way how to do. Only one reason why I write this all is there probably are no software ready for your task.

If you don't know how to write that program, simply ask on stackoverflow.com, not superuser. There are many programmers that are ready for help.

If it's possible, check your ftp site supports download resuming. Whithout this reading end part of file is impossible.

Another partial resolution may be log file rotation, one file a day or a hour. this depends how fast log file grows.

Похожие вопросы