The AVR simply doesn't have the resources to manage a PCI bus, and in no way could attempt a PCIe bus (the serial differential bus is running at 2.5Gbit/s). The total throughput of an AVR is approximately 0.4*clockRate bytes per second, doing nothing but copying one port to another port in a tight loop.
To prototype on a PCI or PCIe bus you need the high speed requirement for your product first off, or you will be causing yourself a ton of work for no reason (kernel drivers, etc). There are upper end FPGAs capable of this.
Digilent sells an FPGA board with a PCI bus and quad GigE Ethernet interfaces, designed for wire-speed packet handling. Its also around $1k.
http://www.digilentinc.com/Products/Det ... od=NETFPGA