Concatenate Two QFileSystemModels

Question

In this post, my goal is to concatenate two QFileSystemModels to one and display them together. (Lots of updates has been made)

Context :

In my C drive , I created the folder MyFolder (https://drive.google.com/drive/folders/1M-b2o9CiohXOgvjoZrAnl0iRVQBD1sXY?usp=sharing) , in which there are some folders and some files, for the sake of producing the minimal reproducible example . Their structure is :

The following Python code using PyQt5 library (modified from How to display parent directory in tree view?) runs after importing necessary libraries:

#The purpose of the proxy model is to display the directory.
#This proxy model is copied here from the reference without modification.
class ProxyModel(QSortFilterProxyModel):
    def __init__(self, parent=None):
        super().__init__(parent)
        self._root_path = ""

    def filterAcceptsRow(self, source_row, source_parent):
        source_model = self.sourceModel()
        if self._root_path and isinstance(source_model, QFileSystemModel):
            root_index = source_model.index(self._root_path).parent()
            if root_index == source_parent:
                index = source_model.index(source_row, 0, source_parent)
                return index.data(QFileSystemModel.FilePathRole) == self._root_path
        return True

    @property
    def root_path(self):
        return self._root_path

    @root_path.setter
    def root_path(self, p):
        self._root_path = p
        self.invalidateFilter()


class MainWindow(QMainWindow):
    def __init__(self, parent=None):
        super().__init__(parent)
        self.create_treeview()
        self.setCentralWidget(self.treeView_1) #The line I will be talking about.

    def create_treeview(self):
        
        self.treeView_1 = QTreeView()
        self.dirModel_1 = QFileSystemModel()
        self.dirModel_1.setRootPath(QDir.rootPath())
        path_1 = 'C:/MyFolder/SubFolder1' # Changing the path is sufficient to change the displayed directory
        root_index_1 = self.dirModel_1.index(path_1).parent()
        self.proxy_1 = ProxyModel(self.dirModel_1)
        self.proxy_1.setSourceModel(self.dirModel_1)
        self.proxy_1.root_path = path_1
        self.treeView_1.setModel(self.proxy_1)
        proxy_root_index_1 = self.proxy_1.mapFromSource(root_index_1)
        self.treeView_1.setRootIndex(proxy_root_index_1)
        
        self.treeView_2 = QTreeView()
        self.dirModel_2 = QFileSystemModel()
        self.dirModel_2.setRootPath(QDir.rootPath())
        path_2 = 'C:/MyFolder'
        root_index_2 = self.dirModel_2.index(path_2).parent()
        self.proxy_2 = ProxyModel(self.dirModel_2)
        self.proxy_2.setSourceModel(self.dirModel_2)
        self.proxy_2.root_path = path_2
        self.treeView_2.setModel(self.proxy_2)
        proxy_root_index_2 = self.proxy_2.mapFromSource(root_index_2)
        self.treeView_2.setRootIndex(proxy_root_index_2)

if __name__ == "__main__":
    import sys

    app = QApplication(sys.argv)
    w = MainWindow()
    w.show()
    sys.exit(app.exec_())

The line self.setCentralWidget(self.treeView_1) gives:

Changing self.setCentralWidget(self.treeView_1) to self.setCentralWidget(self.treeView_2) gives:

Objective:

My goal is to concatenate the two trees together. That is, when click run, the user should be able to see:

The order which they show up does not matter. All I care is that MyFolder and SubFolder1 show up as if they are completely independent items (even though in reality one is a subfolder of the other). I should remark that everything is static. That is, we are not trying to detect any changes on folders or files. The only time we ever need to peak at the existing folders and files will be when we click on run.

Update:

After several days of studying and trying, a major progress has been made. I thank musicamante for the hint of using QTreeWidget. The idea is, as said in comments, traverse through models and gradually move everything into one new QTreeWidget. To avoid freeze, my solution is to ask the QFileSystemModel to fetchMore whenever the user wants to see more (i.e. when the user wants to extend QTreeWidget).

The following code runs and almost solves my problem:

import os
from PyQt5.QtCore import*
from PyQt5.QtWidgets import*
from PyQt5 import QtTest

class To_Display_Folder(QSortFilterProxyModel):
    def __init__(self, disables=False, parent=None):
        super().__init__(parent)
        #self.setFilterRegularExpression(r'^(.*\.dcm|[^.]+)$')
        self._disables = bool(disables)
        
        self._root_path = ""

    def filterAcceptsRow(self, source_row, source_parent):
        source_model = self.sourceModel()
        #case 1 folder
        if self._root_path and isinstance(source_model, QFileSystemModel):
            root_index = source_model.index(self._root_path).parent()
            if root_index == source_parent:
                index = source_model.index(source_row, 0, source_parent)
                return index.data(QFileSystemModel.FilePathRole) == self._root_path
        
        return True
        
'''
        #case 2 file
        file_index = self.sourceModel().index(source_row, 0, source_parent)
        if not self._disables:
            return self.matchIndex(file_index)
        return file_index.isValid()
'''        
    @property
    def root_path(self):
        return self._root_path

    @root_path.setter
    def root_path(self, p):
        self._root_path = p
        self.invalidateFilter()

    def matchIndex(self, index):
        return (self.sourceModel().isDir(index) or
                super().filterAcceptsRow(index.row(), index.parent()))

    def flags(self, index):
        flags = super().flags(index)
        if (self._disables and
            not self.matchIndex(self.mapToSource(index))):
            flags &= ~Qt.ItemIsEnabled
        return flags

class Widget_Item_from_Proxy(QTreeWidgetItem):
    def __init__(self, index_in_dirModel, parent = None):
        super().__init__(parent)
        self.setText(0, index_in_dirModel.data(QFileSystemModel.FileNameRole))
        self.setText(1, index_in_dirModel.data(QFileSystemModel.FilePathRole))
        if os.path.isfile(index_in_dirModel.data(QFileSystemModel.FilePathRole)):
            self.setIcon(0,QApplication.style().standardIcon(QStyle.SP_FileIcon))
        else:
            self.setIcon(0,QApplication.style().standardIcon(QStyle.SP_DirIcon))

class MainWindow(QMainWindow):
    def __init__(self, parent=None):
        super().__init__(parent)
        global treeWidget 
        treeWidget = QTreeWidget()
        self.treeWidget = treeWidget
        self.treeWidget.itemExpanded.connect(self.upon_expansion)
        self.treeWidget.itemClicked.connect(self.tree_click)
        
        #The following directories will be displayed on the tree.
        self.add_path_to_tree_widget('C:/MyFolder')
        self.add_path_to_tree_widget('C:/Users/r2d2w/OneDrive/Desktop')
        self.add_path_to_tree_widget('C:/')
        
        self.setCentralWidget(self.treeWidget)
    
    def add_path_to_tree_widget(self,path):
        dirModel = QFileSystemModel()
        dirModel.setRootPath(QDir.rootPath())
        dirModel.directoryLoaded.connect(lambda: self.once_loaded(path, dirModel))
    
    def once_loaded(self, path, dirModel):
        if dirModel.canFetchMore(dirModel.index(path)):
            dirModel.fetchMore(dirModel.index(path))
            return
        root_index = dirModel.index(path).parent()
        proxy = To_Display_Folder(disables = False, parent = dirModel)
        proxy.setSourceModel(dirModel)
        proxy.root_path = path
        proxy_root_index = proxy.mapFromSource(root_index)
        origin_in_proxy = proxy.index(0,0,parent = proxy_root_index)
        root_item = Widget_Item_from_Proxy(
            proxy.mapToSource(origin_in_proxy))
        self.treeWidget.addTopLevelItem(root_item)
        
        for row in range(0, proxy.rowCount(origin_in_proxy)):
            proxy_index = proxy.index(row,0,parent = origin_in_proxy)
            child = Widget_Item_from_Proxy(
                proxy.mapToSource(proxy_index), 
                parent = self.treeWidget.topLevelItem(self.treeWidget.topLevelItemCount()-1))
        dirModel.directoryLoaded.disconnect()
        
    @pyqtSlot(QTreeWidgetItem)
    def upon_expansion(self, treeitem):
        for i in range(0, treeitem.childCount()):
            if os.path.isdir(treeitem.child(i).text(1)):
                self.add_child_path_to_tree_widget(treeitem.child(i))

    def add_child_path_to_tree_widget(self,subfolder_item):
        subfolder_path = subfolder_item.text(1)
        dirModel = QFileSystemModel()
        dirModel.setRootPath(QDir.rootPath())
        dirModel.directoryLoaded.connect(lambda: self.child_once_loaded(subfolder_item, subfolder_path,dirModel))

    def child_once_loaded(self, subfolder_item, subfolder_path, dirModel):
        if dirModel.canFetchMore(dirModel.index(subfolder_path)):
            dirModel.fetchMore(dirModel.index(subfolder_path))
            return
        
        root_index = dirModel.index(subfolder_path).parent()
        proxy = To_Display_Folder(disables = False, parent = dirModel)
        proxy.setSourceModel(dirModel)
        proxy.root_path = subfolder_path
        proxy_root_index = proxy.mapFromSource(root_index)
        origin_in_proxy = proxy.index(0,0,parent = proxy_root_index)
        
        root_item = Widget_Item_from_Proxy(
            proxy.mapToSource(origin_in_proxy))

        folder_item = subfolder_item.parent()
        itemIndex = folder_item.indexOfChild(subfolder_item)
        folder_item.removeChild(subfolder_item)
        folder_item.insertChild(itemIndex, root_item)

        for row in range(0, proxy.rowCount(origin_in_proxy)):
            proxy_index = proxy.index(row,0,parent = origin_in_proxy)
            child = Widget_Item_from_Proxy(
                proxy.mapToSource(proxy_index), 
                parent = root_item)

        dirModel.directoryLoaded.disconnect()

    @pyqtSlot(QTreeWidgetItem)
    def tree_click(self, item):
        print(item.text(0))
        print(item.text(1))

if __name__ == "__main__":
    import sys

    app = QApplication(sys.argv)
    w = MainWindow()
    w.show()
    sys.exit(app.exec_())

Since the bounty period is still not over, I will use the time to post two new questions:

Sometimes, especially when the line self.add_path_to_tree_widget('C:/') is present, the code does not give all directories when we click run. This problem is easily fixed by closing the window and clicking on run again. This problem occurs because the QFileSystemModel does not yet have enough time to traverse through the designated folder. If it has just a little bit more time, it will be able to. I wonder if there is a way to fix this programatically.
The function add_path_to_tree_widget is similar to add_child_path_to_tree_widget. The function once_loaded is similar to child_once_loaded. I wonder if there is a way to write these functions more succinctly.

Showing the contents of two separate models in a single view is quite difficult. Are the contents of those directories dynamic, or static? Because if they are static, it would be quite simple, otherwise you might need to create a custom QAbstractItemModel, and a quite complex one, especially if you want it to be consistent with selections, content changes (files/directories added/removed/renamed), etc. — musicamante, Jun 13 '22 at 21:11
They are static. They are just existing folders in the computer and they are not changing. It is expected to be simple but I am just not sure about the syntax. — 温泽海, Jun 13 '22 at 21:15
Then, instead of using a QTreeView, you could use a QTreeWidget and add items taken from those two QFileSystemModels using a recursive function. — musicamante, Jun 13 '22 at 21:20
Ah I completely forgot about `QTreeWidget`. I will definitely go have a try. — 温泽海, Jun 13 '22 at 21:24
@jie please write comments in English language: the fact that those comments are intended for the OP is irrelevant, as they would encourage foreign (and not-understandable) communication that could still be useful to others. — musicamante, Jun 14 '22 at 06:55
I just updated the question. It looks like I still need further hints on how to do it. I can concatenate widgets using `QTreeWidget` but I need them to be in one widget. — 温泽海, Jun 14 '22 at 16:42
I believe you misunderstood what I explained: you have to create a *single* QTreeWidget, and add items to it from the two separate models. In fact, you could even ignore the models, and use [QDirIterator](https://doc.qt.io/qt-5/qdiriterator.html) or [`QDir.entryInfoList()`](https://doc.qt.io/qt-5/qdir.html#entryInfoList-1) to create the items. — musicamante, Jun 18 '22 at 00:11
I tried very hard in the past three days. I have created a new treewidget and I get what you mean by adding items recursively. Since my goal is to apply name filters in a way described in https://stackoverflow.com/questions/72587813/how-to-filter-by-no-extension-in-qfilesystemmodel, it seems I need to use models. f I am still stuck on how to add items because I don't know how to get child from an index of `QFileSystemModels` in my setting. — 温泽海, Jun 18 '22 at 01:19
You need to use models as long as the contents are supposed to be **directly** shown in an item view. If you're programmatically creating the items, that's really not a requirement. In fact, you could even just use the basic python SL to browse the filesystem. — musicamante, Jun 18 '22 at 01:30
Your new update seem to have many issues, starting with the fact that you're creating *two* models for each new requested directory. I didn't have time to carefully test that solution yet, but, at first sight, it can create some serious memory issues for very large directory contents (in terms of directory count and sub-directory levels), resulting in serious performance issues. Even if memory wasn't an issue, that's clearly not very effective: you can do that with just *one* QFileSystemModel for top level path, and eventually programmatically filter the contents without using a proxy. — musicamante, Jun 18 '22 at 23:34
But shouldn't the models be forgotten immediately after I have my tree widget? If my intuition on the code is correct, there will be at most one model fetching/storing things at a time, which will immediately be forgotten once the widget is born because the tree widgets do not store the models or indices. When I run it, it seems everything works fine except the (small) problem I described above. I know it is not effective and I have attempted to use just one model (but I was stuck on that approach and eventually switched to this one here). — 温泽海, Jun 19 '22 at 00:11
The problem is quite complex, and results may vary depending on the OS, but adding a basic `dirModel.destroyed.connect(lambda: print('model destroyed'))` within `add_path_to_tree_widget` will probably show that not all models are being destroyed, and the lambdas add even more complexity to the issue (see [this related answer](//stackoverflow.com/a/47945948)). In fact, this may be the actual reason of the issue: the model gets gc before being able to emit the signal. In any case, creating *new models* is certainly not a good idea, especially since the contents of the folders are static. — musicamante, Jun 19 '22 at 01:01
You are right. The models are not deleted. Even if I add the line `del dirModel` in `once_loaded` the model is still not deleted. I don't want to create new models either. But I am stuck on how to use the models that already exist. — 温泽海, Jun 19 '22 at 14:47

score 0 · Answer 1 · answered Jun 18 '22 at 03:32

While not impossible, it's quite difficult to create a unique and dynamic model that is able to access different QFileSystemModel structures.

An easier and simpler implementation, which would be more practical for static purposes, is to use a QTreeWidget and create items recursively.

class MultiBrowser(QTreeWidget):
    def __init__(self, *pathList):
        super().__init__()
        self.iconProvider = QFileIconProvider()
        self.setHeaderLabels(['Name'])

        for path in pathList:
            item = self.createFSItem(QFileInfo(path), self.invisibleRootItem())
            self.expand(self.indexFromItem(item))

    def createFSItem(self, info, parent):
        item = QTreeWidgetItem(parent, [info.fileName()])
        item.setIcon(0, self.iconProvider.icon(info))
        if info.isDir():
            infoList = QDir(info.absoluteFilePath()).entryInfoList(
                filters=QDir.AllEntries | QDir.NoDotAndDotDot, 
                sort=QDir.DirsFirst
            )
            for childInfo in infoList:
                self.createFSItem(childInfo, item)
        return item

# ...
multiBrowser = MultiBrowser('path1', 'path2')

For obvious reasons, the depth of each path and their contents will freeze the UI from interaction until the whole structure has been crawled.

If you need a more dynamic approach, consider using the QFileSystemModel as a source for path crawling, along with its directoryLoaded signal, which will obviously require a more complex implementation.

I just made a major progress to deal with the freeze problem. See the update part. — 温泽海, Jun 18 '22 at 20:33

Concatenate Two QFileSystemModels

1 Answers1