PCIE region not aligned, and not consistent

Question

I am developing a PCIE device driver for openwrt, and I met a data bus error when trying to access the io-memory in timer interrupt, which I mentioned in my last question. After lots of research I think I might have found the reason, but I am unable to solve it. Below are my troubles.

Last week I found out that the pcie region size might have changed during system startup. The region size of bar0 is 4096 in my driver (return from pci_resource_len) and the region size is 4097 in lspci -vv, which breaks the page size of linux kernel. By reading the source code of pciutil, I find that lspci command fetch the pcie information from /sys/devices/pci0000:00/0000:00:00.0/resouce file. So I remove all my custom components and run the original openwrt on my router. By cat /sys/devices/pci0000:00/0000:00:00.0/resouce, the first line of the result (bar0) is

0x0000000010008000 0x0000000010009000 0x0000000000040200

Moreover, I also check the content of /proc/iomem, and the content related to PCIE is

10000000-13ffffff : mem_base
    10000000-13ffffff : PCI memory space
        10000000-10007fff : 0000:00:00.0
        10008000-10008fff : 0000:00:00.0

It is super weird that the region size of bar0 indicated by the two files above is different! According to the mechanism of PCIE, the region size should always be the power of 2. How come the region size becomes 4097?

score 0 · Answer 1 · answered Nov 14 '17 at 05:56

After spending weeks reading source code of linux kernel, I find out that this is a bug of linux kernel 4.4.14.

The content of /sys/devices/pci0000:00/0000:00:00.0/resouce is generated through function resource_show in file drivers/pci/pci-sysfs.c. The related code is

for (i = 0; i < max; i++) {
    struct resource *res =  &pci_dev->resource[i];
    pci_resource_to_user(pci_dev, i, res, &start, &end);
    str += sprintf(str, "0x%016llx 0x%016llx 0x%016llx\n",
               (unsigned long long)start,
               (unsigned long long)end,
               (unsigned long long)res->flags);
}

The function pci_resource_to_user actually invoked is located in arch/mips/include/asm/pci.h

static inline void pci_resource_to_user(const struct pci_dev *dev, int bar,
        const struct resource *rsrc, resource_size_t *start,
        resource_size_t *end)
{
    phys_addr_t size = resource_size(rsrc);

    *start = fixup_bigphys_addr(rsrc->start, size);
    *end = rsrc->start + size;
}

The calculation of *end is wrong and should be replace by

*end = rsrc->start + size - (size ? 1 : 0)

PCIE region not aligned, and not consistent

1 Answers1