This is due to the order of the directives, once Apache reads the combined configuration file.
Apache will use the first matching site ServerName or ServerAlias it finds, so if your wildcard host appears first in the config files then apache will match using that and never see the special config (because special.domain.com also matches *.domain.com). So - you need to make sure the special.domain.com conf file is loaded and parsed first so that apache will choose it as the first match. Only sites that don't make that first match will then "fall through" and match *.domain.com instead.
On Ubuntu, Debian, or other Debian-based systems, these files are found in /etc/apache2/sites-enabled (these are in fact symbolic links to the actual .conf files in /etc/apache2/sites-available). Apache will read the files in that directory in alphabetical order, so you can force a file to read first by renaming its link to "001-something.conf" rather than "something.conf". Debian uses this trick by default - it sets up the default website as "001-default.conf" (linked to ../sites-available/default.conf). This causes the default virtualhost to load first so it's the one you see if nothing later matches. Just like you can name a conf file 001-something.conf you could also name it: ZZ-something.conf to make it load last. In all these cases you are renaming the files in /etc/apache2/sites-enabled (not /etc/apache2/sites-available).
Under Redhat / Centos / Fedora, you combine vhosts into a single configuration file. If that's your environment then just re-order them until they work, following instructions below.
To fix your problem experiment with renaming your conf files in /etc/apache2/sites-available so that your wildcard site loads last (Rename it to start with ZZ-). Or, if you determine it's loading last already, try making it load first (rename it to start with 001 or 002- or something). One of them should fix it and it will behave as you expect.