PCI 驱动编程基础

lspci

lspci 可以列出系统中所有的 PCI 设备。

$ lspci
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE (rev 01)
00:02.0 VGA compatible controller: VMware SVGA II Adapter
00:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)
00:04.0 System peripheral: InnoTek Systemberatung GmbH VirtualBox Guest Service
00:05.0 Multimedia audio controller: Intel Corporation 82801AA AC’97 Audio Controller (rev 01)
00:07.0 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08)
00:0c.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller
00:0d.0 SATA controller: Intel Corporation 82801HM/HEM (ICH8M/ICH8M-E) SATA Controller [AHCI mode] (rev 02)

以加粗那行 PCI 设备为例
00 是 Bus ID
03 是 Device ID
0 是 Function ID

lspci -v 显示设备详细的信息

$ lspci -v
00:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)
Subsystem: Intel Corporation PRO/1000 MT Desktop Adapter
Flags: 66MHz, medium devsel, IRQ 19
Memory at f0200000 (32-bit, non-prefetchable) [size=128K]
I/O ports at d020 [size=8]
Capabilities:
Kernel driver in use: e1000
Kernel modules: e1000

还是以上面的 PCI 设备为例:

• Flags: 66MHz, medium devsel, IRQ 19：中断号为 19
• Memory at f0200000：地址为 f0200000
• Kernel driver in use: e1000 ：表示该设备使用的驱动为 e1000

lspci -x 以十六进制显示 PCI 配置空间 (configuration space) 的前64个字节映象 (标准头部信息)

$ lspci -x
00:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)
00: 86 80 0e 10 03 00 30 02 02 00 00 02 00 40 00 00
10: 00 00 20 f0 00 00 00 00 21 d0 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 1e 00
30: 00 00 00 00 dc 00 00 00 00 00 00 00 09 01 ff 00

lspci -vvv 显示更详细的信息

00:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)
Subsystem: Intel Corporation PRO/1000 MT Desktop Adapter
Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- Interrupt: pin A routed to IRQ 19
Region 0: Memory at f0200000 (32-bit, non-prefetchable) [size=128K]
Region 2: I/O ports at d020 [size=8]
Capabilities:
Kernel driver in use: my_pci_driver
Kernel modules: e1000

支持两个 BAR 地址空间，Region 0 和 Region 2。

配置空间

标准头信息占 64 字节，内容如下

对照 lspci -x 显示的信息，整理如下

配置寄存器地址空间，64 字节头

#define PCI_STD_HEADER_SIZEOF    64

#define PCI_VENDOR_ID         0x00    /* 16 bits */
#define PCI_DEVICE_ID         0x02    /* 16 bits */

#define PCI_COMMAND           0x04    /* 16 bits */
#define PCI_STATUS            0x06    /* 16 bits */

#define PCI_CLASS_REVISION    0x08    /* High 24 bits are class, low 8 revision */
#define PCI_REVISION_ID       0x08    /* Revision ID */
#define PCI_CLASS_PROG        0x09    /* Reg. Level Programming Interface */

#define PCI_CACHE_LINE_SIZE   0x0c    /* 8 bits */
#define PCI_LATENCY_TIMER     0x0d    /* 8 bits */
#define PCI_HEADER_TYPE       0x0e    /* 8 bits */
#define PCI_BIST              0x0f    /* 8 bits */

#define PCI_BASE_ADDRESS_0    0x10    /* 32 bits */
#define PCI_BASE_ADDRESS_1    0x14    /* 32 bits [htype 0,1 only] */
#define PCI_BASE_ADDRESS_2    0x18    /* 32 bits [htype 0 only] */
#define PCI_BASE_ADDRESS_3    0x1c    /* 32 bits */
#define PCI_BASE_ADDRESS_4    0x20    /* 32 bits */
#define PCI_BASE_ADDRESS_5    0x24    /* 32 bits */

#define PCI_CARDBUS_CIS        0x28
#define PCI_SUBSYSTEM_VENDOR_ID 0x2c
#define PCI_SUBSYSTEM_ID      0x2e
#define PCI_ROM_ADDRESS       0x30    /* Bits 31..11 are address, 10..1 reserved */

#define PCI_CAPABILITY_LIST   0x34    /* Offset of first capability list entry */

#define PCI_INTERRUPT_LINE    0x3c    /* 8 bits */
#define PCI_INTERRUPT_PIN     0x3d    /* 8 bits */
#define PCI_MIN_GNT           0x3e    /* 8 bits */
#define PCI_MAX_LAT           0x3f    /* 8 bits */
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

PCI 驱动 demo

环境：Ubuntu 20.04 虚拟机，有个 Intel 网卡，也就是上面例子中的 00:03.0 设备，默认使用的驱动是 e1000，我们写个 demo，"驱动"这个设备，所以先将原驱动卸载

$ sudo rmmod e1000
1

my_pci.c

#include 
#include 

#define DRV_NAME "my_pci_driver"

struct pci_card {
	// 端口读写变量
	resource_size_t io_start;
	resource_size_t io_end;
	long range;
	long flags;
	void __iomem *ioaddr;
	int irq;
};

void my_pci_get_configs(struct pci_dev *dev)
{
	uint8_t val1;
	uint16_t val2;
	uint32_t val4;

	pci_read_config_word(dev, PCI_VENDOR_ID, &val2);
	printk("vendor id = 0x%x\n", val2);

	pci_read_config_word(dev, PCI_DEVICE_ID, &val2);
	printk("device id = 0x%x\n", val2);

	pci_read_config_byte(dev, PCI_REVISION_ID, &val1);
	printk("revision id = 0x%x\n", val1);

	pci_read_config_dword(dev, PCI_CLASS_REVISION, &val4);
	printk("class = 0x%x\n", val4 >> 8);

	pci_read_config_byte(dev, PCI_CACHE_LINE_SIZE, &val1);
	printk("cache line size = 0x%x\n", val1);

	pci_read_config_byte(dev, PCI_LATENCY_TIMER, &val1);
	printk("latency time = 0x%x\n", val1);

	pci_read_config_byte(dev, PCI_HEADER_TYPE, &val1);
	printk("header type = 0x%x\n", val1);

	pci_read_config_byte(dev, PCI_BIST, &val1);
	printk("bist = 0x%x\n", val1);

	pci_read_config_dword(dev, PCI_BASE_ADDRESS_0, &val4);
	printk("base address 0 = 0x%x\n", val4);

	pci_read_config_dword(dev, PCI_BASE_ADDRESS_1, &val4);
	printk("base address 1 = 0x%x\n", val4);

	pci_read_config_dword(dev, PCI_BASE_ADDRESS_2, &val4);
	printk("base address 2 = 0x%x\n", val4);

	pci_read_config_dword(dev, PCI_BASE_ADDRESS_3, &val4);
	printk("base address 3 = 0x%x\n", val4);

	pci_read_config_dword(dev, PCI_BASE_ADDRESS_4, &val4);
	printk("base address 4 = 0x%x\n", val4);

	pci_read_config_dword(dev, PCI_BASE_ADDRESS_5, &val4);
	printk("base address 5 = 0x%x\n", val4);

	pci_read_config_dword(dev, PCI_CARDBUS_CIS, &val4);
	printk("card bus cis = 0x%x\n", val4);

	pci_read_config_word(dev, PCI_SUBSYSTEM_VENDOR_ID, &val2);
	printk("subsystem vendor id = 0x%x\n", val2);

	pci_read_config_word(dev, PCI_SUBSYSTEM_ID, &val2);
	printk("subsystem id = 0x%x\n", val2);

	pci_read_config_byte(dev, PCI_CAPABILITY_LIST, &val1);
	printk("capability list = 0x%x\n", val1);

	pci_read_config_byte(dev, PCI_INTERRUPT_LINE, &val1);
	printk("interrupt line = 0x%x\n", val1);

	pci_read_config_byte(dev, PCI_INTERRUPT_PIN, &val1);
	printk("interrupt pin = 0x%x\n", val1);

	pci_read_config_byte(dev, PCI_MIN_GNT, &val1);
	printk("min gnt = 0x%x\n", val1);

	pci_read_config_byte(dev, PCI_MAX_LAT, &val1);
	printk("max lat = 0x%x\n", val1);
}

static irqreturn_t my_pci_interrupt(int irq, void *dev_id)
{
	struct pci_card *mypci = (struct pci_card *)dev_id;
	printk("irq = %d, mypci_irq = %d\n", irq, mypci->irq);

	return IRQ_HANDLED;
}

static int probe(struct pci_dev *pci_dev, const struct pci_device_id *ent)
{
	int rc;
	struct pci_card *my_pci = NULL;

	printk("%s() line:%d\n", __FUNCTION__, __LINE__);

	rc = pci_enable_device(pci_dev);
	if (rc) {
		printk("pci enable device failed!\n");
		goto out_mypci;
	}

	my_pci = kmalloc(sizeof(struct pci_card),
					 GFP_KERNEL);  // GFP_KERNEL 是内核内存分配时最常用的，无内存可用时可引起休眠。
	if (my_pci == NULL) {
		printk("kmalloc error!\n");
		rc = -ENOMEM;
		goto out_mypci;
	}

	my_pci->irq = pci_dev->irq;
	if (my_pci->irq < 0) {
		printk("IRQ is %d, it is invalid\n", my_pci->irq);
		goto out_mypci;
	}

	my_pci->io_start = pci_resource_start(pci_dev, 0);	// bar0
	my_pci->io_end = pci_resource_end(pci_dev, 0);
	my_pci->range = my_pci->io_end - my_pci->io_start + 1;
	my_pci->flags = pci_resource_flags(pci_dev, 0);
	printk("io_start = 0x%llx\n", my_pci->io_start);
	printk("io_end = 0x%llx\n", my_pci->io_end);
	printk("range = 0x%lx\n", my_pci->range);
	printk("flags = 0x%lx\n", my_pci->flags);
	printk("PCI base addr 0 is io%s.\n", (my_pci->flags & IORESOURCE_MEM) ? "mem" : "port");

	rc = pci_request_regions(pci_dev, DRV_NAME);
	if (rc) {
		printk("pci request regions error!\n");
		goto out_mypci;
	}

	my_pci->ioaddr = pci_ioremap_bar(pci_dev, 0);
	if (my_pci->ioaddr == NULL) {
		printk("ioremp error!\n");
		rc = ENOMEM;
		goto out_regions;
	}

	rc = request_irq(my_pci->irq, my_pci_interrupt, IRQF_SHARED, DRV_NAME, my_pci);
	if (rc) {
		printk(KERN_ERR "can't get assigned IRQ %d!\n", my_pci->irq);
		goto out_iounmap;
	}

	pci_set_drvdata(pci_dev, my_pci);
	printk("probe success. PCI ioport addr start addr 0x%llx, ioaddr is 0x%p, interrupt No. %d\n",
		   my_pci->io_start, my_pci->ioaddr, my_pci->irq);
	my_pci_get_configs(pci_dev);

	return 0;

out_iounmap:
	iounmap(my_pci->ioaddr);
out_regions:
	pci_release_regions(pci_dev);
out_mypci:
	if (my_pci)
		kfree(my_pci);

	return rc;
}

static void remove(struct pci_dev *pci_dev)
{
	struct pci_card *my_pci;

	printk("%s() line:%d\n", __FUNCTION__, __LINE__);

	my_pci = pci_get_drvdata(pci_dev);
	if (my_pci) {
		free_irq(my_pci->irq, my_pci);
		iounmap(my_pci->ioaddr);
		kfree(my_pci);
	}

	pci_release_regions(pci_dev);

	pci_disable_device(pci_dev);

	printk("device is removed successfully\n");
}

static const struct pci_device_id my_pci_tbl[] = {
	{PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x100e)},
};

static struct pci_driver mv_pci_driver = {
	.name = DRV_NAME,
	.id_table = my_pci_tbl,
	.probe = probe,
	.remove = remove,
};

static int my_pci_init(void)
{
	printk("%s() line:%d\n", __FUNCTION__, __LINE__);

	return pci_register_driver(&mv_pci_driver);
}

static void my_pci_exit(void)
{
	printk("%s() line:%d\n", __FUNCTION__, __LINE__);

	pci_unregister_driver(&mv_pci_driver);
}

module_init(my_pci_init);
module_exit(my_pci_exit);
MODULE_LICENSE("GPL");
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218

Makefile

obj-m = my_pci.o

all:
	make -C /lib/modules/$(shell uname -r)/build/ M=${PWD} modules
clean:
	make -C /lib/modules/$(shell uname -r)/build/ M=${PWD} clean

1
2
3
4
5
6
7

$ sudo rmmod e1000
$ sudo insmod my_pci.ko
$ dmesh
[48344.049257] my_pci_init() line:204
[48344.049285] probe() line:102
[48344.049746] io_start = 0xf0200000
[48344.049748] io_end = 0xf021ffff
[48344.049749] range = 0x20000
[48344.049749] flags = 0x40200
[48344.049750] PCI base addr 0 is iomem.
[48344.049807] probe success. PCI ioport addr start addr 0xf0200000, ioaddr is 0x000000000e21ae12, interrupt No. 19
[48344.049817] vendor id = 0x8086
[48344.049823] device id = 0x100e
[48344.049830] revision id = 0x2
[48344.049837] class = 0x20000
[48344.049843] cache line size = 0x0
[48344.049850] latency time = 0x40
[48344.049856] header type = 0x0
[48344.049863] bist = 0x0
[48344.049869] base address 0 = 0xf0200000
[48344.049876] base address 1 = 0x0
[48344.049882] base address 2 = 0xd021
[48344.049888] base address 3 = 0x0
[48344.049895] base address 4 = 0x0
[48344.049901] base address 5 = 0x0
[48344.049908] card bus cis = 0x0
[48344.049914] subsystem vendor id = 0x8086
[48344.049921] subsystem id = 0x1e
[48344.049927] capability list = 0xdc
[48344.049933] interrupt line = 0x9
[48344.049940] interrupt pin = 0x1
[48344.049946] min gnt = 0xff
[48344.049953] max lat = 0x0

$ sudo rmmod my_pci
[51661.250413] my_pci_exit() line:211
[51661.250433] remove() line:175
[51661.250939] device is removed successfully
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

可以看到，驱动代码从配置空间读到的内容，和 lspci -x 显示的一致，说明驱动程序编写正确，并且通过 pci_ioremap_bar() 将 bar0 的地址（0xf0200000）映射到了虚拟内存地址（0x000000000e21ae12），之后就可以通过这个地址操作 bar0 寄存器了。
不过，这个 “PCI 驱动” 几乎不具备任何实际能力，这里仅作为一个演示，带领大家步入 PCI 驱动的大门。

sysfs && proc

在 sysfs 中找到对应的 PCI 设备，查看其 resource，该文件返回它的 6 个 BAR 信息，按理说应该只有 6 行才对，不知道为啥打印了 13 行。
每个 BAR 里面记录了一片 Memory Map 或 I/O Map

$ cat /sys/bus/pci/devices/0000:00:03.0/resource
0x00000000f0200000 0x00000000f021ffff 0x0000000000040200
0x0000000000000000 0x0000000000000000 0x0000000000000000
0x000000000000d020 0x000000000000d027 0x0000000000040101
0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 0x0000000000000000

上面前 6 行对应 6 个 BAR，
第一行是 BAR0，记录的是一个 Memory Map，通过读取 /proc/iomem 可以佐证，如下

$ sudo cat /proc/iomem 
...
  f0200000-f021ffff : 0000:00:03.0
  	f0200000-f021ffff : my_pci_driver
...
1
2
3
4
5

第三行是 BAR2，记录的是一个 I/O Map，通过读取 /proc/ioports 可以佐证，如下

$ sudo cat /proc/ioports 
...
  d020-d027 : 0000:00:03.0
    d020-d027 : my_pci_driver
...
1
2
3
4
5