Python watch QNAP FTP directory for changes and send UDP messages

Question

I need to write a program that watches a directory on ftp server and then sends a message with the path to the new file. So I did manage to use watchdog for a local folder with this as I need just the create event:

if __name__ == "__main__":
    patterns = "*"
    ignore_patterns = ""
    ignore_directories = False
    case_sensitive = False
    my_event_handler = PatternMatchingEventHandler(patterns, ignore_patterns, ignore_directories, case_sensitive)

    def on_created(event):
        byte_message = bytes(f"{event.src_path}", "utf-8")
        opened_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        opened_socket.sendto(byte_message, ('address', port))
        print(f"{event.src_path} created")
    
    my_event_handler.on_created = on_created

    path = r"local/path"
    go_recursively = True
    my_observer = Observer()
    my_observer.schedule(my_event_handler, path, recursive=go_recursively)

    my_observer.start()
    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        my_observer.stop()

But when I try replacing the path with the folder on that FTP I get Access Denied which is correct as I did't set the login and the password. After that I did this, based on an answer on Stack Overflow:

ftp = FTP()
ftp.set_pasv(True)
ftp.connect("address")
ftp.login('user', 'pass')

def changemon(dir='ftp/path/*'):
    ls_prev = set()

    while True:
        ls = set(ftp.nlst(dir))

        add = ls-ls_prev
        if add: 
            yield add 
        ls_prev = ls
        sleep(5)

for add in changemon():
    byte_message = bytes('\n'.join(r'address%' % i for i in add), 'utf-8')
    opened_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    opened_socket.sendto(byte_message, ('address', port))

The problem with this is that it gets every single file that appears in the dir.

So I need something that I can extract the subfolder names and file names from. Like this: file1 and file2 are newly created:

ftp/path/5000/file1
ftp/path/5001/file2

print('Folder is: ' + foldername)
Folder is: 5000
Folder is: 5001
print('New file is: ' + filename)
New file is: file1
New file is: file2

Any help is welcome.

The first time it runs it will always return every file, because your `ls_prev` will be empty. So if this code is inside a process that ends and restarts, you'll have to find a way to save the value of `ls_prev` and read it back before entering the loop. A simple pickle should work just fine for that. Keep in mind that this will not detect a file being changed (a new version of the same file name uploaded). — ChatterOne, Apr 22 '21 at 13:58
@T0ny1234 this causes a lot of noise, you are sure you only can run that code remotely ? — user3732793, Apr 23 '21 at 07:25
@user3732793 Yes, it's for monitoring an QNAP ftp directories. And this is the only way I figured how to do the task. If you have the time can you suggest a better way? — Tony, Apr 23 '21 at 07:29
depends what qnap you have. looks like some have SNMP interfaces. Here a icinga plugin https://github.com/Mikesch-mp/qnap_health — user3732793, Apr 23 '21 at 07:32
@user3732793 Thanks I didn't know that existed. Can it be used for the result I want to achieve i.e watch directory and send udp/tcp message? — Tony, Apr 23 '21 at 07:39
SNMP is used for network devices management. So yes you can write a script loop over these calls and the device tells you what is going on..some learn curve is involved — user3732793, Apr 23 '21 at 07:55

score 0 · Accepted Answer · answered Jun 08 '21 at 06:50

This is the working solution after me not able to find anything that could help me from the store (and the task was to make it in python):

from ftplib import FTP
from time import sleep
import os
import byte
import socket
import numpy as np
import pandas as pd

ftp = FTP()
ftp.set_pasv(True)
ftp.connect("some-ip-address")
ftp.login('user', 'password')

def changemon(dir='/dir/*'):
    ls_prev = set()

    while True:
        ls = set(ftp.nlst(dir))

        add = ls-ls_prev
        if add:
            yield add

        ls_prev = ls
        sleep(5)

#tcp
for add in changemon():
    result = [i for i in add]
    array = np.array(result, dtype='str')
    df = pd.DataFrame(array, columns=None, index=None)
    for a in array:
        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        s.connect((host, port))
        name = a.split('/')
        separator = ''
        path = (r'ip\dir\\' + str(name[2]) + '\\' + str(name[3]))
        device = name[2]
        message = bytes(separator.join(f'{device} desired string for the IP notification  \n'), 'utf-8')
        s.send(message)
        sleep(5)

It's not the prettiest solution, but gets the job done. The slashes are in the oposite direction because the message needs to be read in Windows. The path variable is then inserted in a database.

Python watch QNAP FTP directory for changes and send UDP messages

1 Answers1