I recently found the Group policy of our domain (windows server 2012) is not updated enough to support our Win10 end-user machine. which are the correct ways to update the Group policy definition library? a. manually install admx files? b. install one domain controller with server 2016? c. raise the function level of the domain to the server 2016?
Don't raise domain functional level. If you do, the Server 2012 DCs will no longer be able to service the domain.
My recommendation would be to create a central store. The central store can be created on the SYSVOL partition on any domain controller (SYSVOL replicates through DFS to all domain controllers) and then go to the domain folder and find the Policies folder. An easy way to access this would be to use the following UNC path: "\FQDN of domain\SYSVOL\FQDN of domain\Policies".
From there, presuming that a PolicyDefinitions folder has not yet been configured, none will exist. Go to a Server 2016 DC and copy the C:\Windows\PolicyDefinitions folder to the aforementioned directory.
This will copy all of the ADMX files over to the central store where all DCs can access them, including Server 2012 DCs.
When I had Windows 2012 R2 server and Windows 10 clients, I had to download ADMX templates and place them in server policy store with each Windows 10 major version.
This link should help you: