You should learn a little bit about USB device classes. CDC is a USB device class, and ACM is a subclass that I assume you are using. The device you made could be called a "CDC ACM device" because it uses the CDC class and the ACM subclass.
These classes and subclasses are defined by the USB Implementers Forum in documents that you can find here:
http://www.usb.org/developers/docs/devclass_docs/
These documents specify things like what USB descriptors a CDC ACM device should have in order to describe itself to the host, and what kinds of interfaces and endpoints it should have, and how serial data will be represented in terms of USB transactions and transfers.
Note that CDC ACM only specifies some USB commands for transferring data between the host and the device. It does not specify what the device will actually do with that data. You can use CDC ACM to implement a USB-to-serial adapter, or you can just use it as a general purpose communication interface for whatever data you want to send.
Yes, you do need a driver on the PC side. The driver needs to be designed to run on your specific operating system. It needs to create some kind of virtual serial port device in your operating system that other software (which only knows about serial ports) can find and connect to. It needs to translate serial port operations performed by other software on the serial port (e.g. writing some bytes to the serial port) into low-level USB commands according to the CDC ACM specifications (e.g. sending some bytes out to the device on a particular endpoint in the form of USB packets). It needs to somehow know which USB devices it should operate on, since not every USB device is a CDC ACM device.
For Windows, you will probably use the usbser.sys driver which comes with Windows. For versions of Windows older than Windows 10, you will need to write an INF file to associate your device to usbser.sys and sign it. For Windows 10 and later, there is a new INF file called usbser.inf already included with Windows which will automatically match any valid CDC ACM device. This means you don't have to write or distribute a driver for CDC ACM devices if you only intend to support using the device on Windows 10 or later. The partnership between Microsoft and Arduino which began in 2015 gives me hope that Microsoft will continue supporting and improving usbser.sys in the future. In fact, they claim that in Windows 10 "the driver has been rewritten by using the Kernel-Mode Driver Framework that improves the overall stability of the driver", so that is good news.
For Linux, there is the cdc_acm kernel module, which has been a standard part of the kernel for a long time and should work automatically with any CDC ACM device you plug in.
For Mac OS X, there is the AppleUSBCDCACM driver, which should work automatically with any CDC ACM device you plug in.
Note that for any of these drivers to recognize your device and work with it, your device has to have certain values in its USB descriptors, and the requirements can vary depending on what exact driver version you are talking about.
Will the USB port now simply look like a serial port?
No, that's the wrong way to think about it. The USB port will still look like a USB port, but the various USB drivers provided by your operating system will recognize that a CDC ACM device is plugged into that port and create a new entry in your operating system's list of serial ports. Then if you run some software that only knows about serial ports, it can connect to that port.
In fact, if you make a composite device, you can have a single USB device plugged into a single USB port that actually has two or more virtual serial ports.