Making libmagic/file detect .docx files

Question

As seen elsewhere, docx, xlsx and pttx are ZIPs. When uploading them to my web application, file (via libmagic andpython-magic) detects them as being ZIP.

I store the contents of the file as a blob in the database, but naturally I don't want to trust the user with what kind of file type this is. So I would like to trust file for and automatically generate a filename during download.

I know one can modify /etc/magic but the format (magic(5)) is way too complicated for me. I found a bug report on the issue at Debian bugs but since it's from 2008 it doesn't seem to be fixed any time soon.

I guess my only other alternative is to indeed trust the user (but still store the contents as a blob) and only check the file extension based on the file name. This way I can disallow some extensions and allow others. And when the user re-downloads his file, he can have it in whatever way he uploaded it. But this solution is insecure if the file is shared with others, since you can simply rename the file to allow uploading it.

Any ideas?

Lastly, I found a list of magic numbers for docx etc, but I'm unable to convert these into the magic(5) format.

Answer 1

You can use

0       string  PK\x03\x04\x14\x00\x06\x00      Microsoft Office Open XML Format

in /etc/magic to identify the general file type based on the information you supplied.

(However, this might not be universal: PK\x03\x04\x00\x14\x08\x08 has been observed at the start of LibreOffice-generated XLSX files.)

Later versions of Ubuntu have a go at correctly identifying the .docx, .pptx, and .xlsx files. Digging around in the sorce code for the file utility I found the ~/file-5.09/magic/Magdir/msooxml file which does the identification. You can get a copy of the file and add it to your /etc/magic file.

---

Including copy of the file that has been updated to v 1.5

---

# $File: msooxml,v 1.5 2014/08/05 07:38:45 christos Exp $
# msooxml:  file(1) magic for Microsoft Office XML
# From: Ralf Brown <[email protected]>

# .docx, .pptx, and .xlsx are XML plus other files inside a ZIP
#   archive.  The first member file is normally "[Content_Types].xml".
#   but some libreoffice generated files put this later. Perhaps skip
#   the "[Content_Types].xml" test?
# Since MSOOXML doesn't have anything like the uncompressed "mimetype"
#   file of ePub or OpenDocument, we'll have to scan for a filename
#   which can distinguish between the three types

# start by checking for ZIP local file header signature
0       string      PK\003\004
!:strength +10
# make sure the first file is correct
>0x1E       regex       \\[Content_Types\\]\\.xml|_rels/\\.rels
# skip to the second local file header
# since some documents include a 520-byte extra field following the file
# header, we need to scan for the next header
>>(18.l+49) search/2000 PK\003\004
# now skip to the *third* local file header; again, we need to scan due to a
# 520-byte extra field following the file header
>>>&26      search/1000 PK\003\004
# and check the subdirectory name to determine which type of OOXML
# file we have.  Correct the mimetype with the registered ones:
# http://technet.microsoft.com/en-us/library/cc179224.aspx
>>>>&26     string      word/       Microsoft Word 2007+
!:mime application/vnd.openxmlformats-officedocument.wordprocessingml.document
>>>>&26     string      ppt/        Microsoft PowerPoint 2007+
!:mime application/vnd.openxmlformats-officedocument.presentationml.presentation
>>>>&26     string      xl/     Microsoft Excel 2007+
!:mime application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
>>>>&26     default     x       Microsoft OOXML
---

But leaving V1.2 here for posterity.

Including a copy here as the link above can go out of date as the file package is updated.

#------------------------------------------------------------------------------
# $File: msooxml,v 1.2 2013/01/25 23:04:37 christos Exp $
# msooxml:  file(1) magic for Microsoft Office XML
# From: Ralf Brown <[email protected]>

# .docx, .pptx, and .xlsx are XML plus other files inside a ZIP
#   archive.  The first member file is normally "[Content_Types].xml".
# Since MSOOXML doesn't have anything like the uncompressed "mimetype"
#   file of ePub or OpenDocument, we'll have to scan for a filename
#   which can distinguish between the three types

# start by checking for ZIP local file header signature
0               string          PK\003\004
# make sure the first file is correct
>0x1E           string          [Content_Types].xml
# skip to the second local file header
#   since some documents include a 520-byte extra field following the file
#   header,  we need to scan for the next header
>>(18.l+49)     search/2000     PK\003\004
# now skip to the *third* local file header; again, we need to scan due to a
#   520-byte extra field following the file header
>>>&26          search/1000     PK\003\004
# and check the subdirectory name to determine which type of OOXML
#   file we have
#   Correct the mimetype with the registered ones:
#     http://technet.microsoft.com/en-us/library/cc179224.aspx
>>>>&26         string          word/           Microsoft Word 2007+
!:mime application/vnd.openxmlformats-officedocument.wordprocessingml.document
>>>>&26         string          ppt/            Microsoft PowerPoint 2007+
!:mime application/vnd.openxmlformats-officedocument.presentationml.presentation
>>>>&26         string          xl/             Microsoft Excel 2007+
!:mime application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
>>>>&26         default         x               Microsoft OOXML
!:strength +10

Answer 2

file, version prior to 5.13, will truncate MIME type to 64 characters. So using the content of msooxml, the MIME type from file -bi command becomes "mime application/vnd.openxmlformats-officedocument.wordprocessingml.d; charset=binary"

Answer 3

if use libreoffice's docx，you can add content(below) to /etc/magic:

# start by checking for ZIP local file header signature
0               string          PK\003\004
!:strength +10
>1104           search/300      PK\003\004
# and check the subdirectory name to determine which type of OOXML
# file we have.  Correct the mimetype with the registered ones:
# http://technet.microsoft.com/en-us/library/cc179224.aspx
>>&26           string          word/           Microsoft Word 2007+
!:mime application/vnd.openxmlformats-officedocument.wordprocessingml.document
>>&26         string          ppt/            Microsoft PowerPoint 2007+
!:mime application/vnd.openxmlformats-officedocument.presentationml.presentation
>>&26         string          xl/             Microsoft Excel 2007+
!:mime application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
>>&26         default         x               Microsoft OOXML