Is it possible if GPT 4o or GPT 4 extract content in the documents (pdf, word,excel) if user upload it in the chatbot?

@Farah Diana Masri AFAIK the GPT models use images to extract text that was uploaded to the chat. They do not directly support files like PDF or word or excel. If you are looking for a functionality that needs to extract correct or exact data from a document, you should consider Azure Document intelligence instead. GPT models can still read text from uploaded JPEG images, but it might not preserve the structure of the document in the provided result. A service like document intelligence will provide co-ordinates and accurate text along with support for common documents with prebuilt models that can be used on the fly.

However, there is a sample repo to showcase using GPT models with document extraction in this repo. This features a sample to use GPT-4o model which converts an uploaded PDF to image files using 3rd party libraries. This might work well with smaller document files to store the context in the conversation but for larger files it might not work as intended.

I hope this helps!!

Thanks for your reply. Is the Document Intelligence is capable to extract the content from the documents like pdf, word and excel that user upload? Because, currently the GPT model like 4o we are not able to upload documents, but why in the ChatGPT we can upload the documents in the chatbot?

Do we need to integrate Document Intelligence with GPT 4o or GPT4 in order for us to upload documents in the chatbot and extract the text and send back to the user?

Thanks for the reply. In order to extract the text from the uploaded documents in the chatbot, is it we need to integrate the Document Intelligence with our GPT4o model?

HI @Farah Diana Masri

You can try to use Add your data option in the Chat playground to start understanding how the document is processed.

Usually, when user attaches a document in the chat, the document will need to be processed and parse into text under the hood (probably this is what chatgpt does). It is not done by the GPT model but an ingestion service: in this case, Azure AI Search provides options to ingest pdf, excel, ppt documents into vector database. Then during the GPT call, these documents can be referenced and used to generate response.

For images, the GPT module has the vision feature that is able to process text in the images directly.

Yes, my previous question is about the second scenario. Thanks for clarifying. Just want to confirm only images is supported for GPT to give context based on the images user upload in the chatbot. However, for documents like (pdf, word or pptx) format, the GPT in Azure does not have the capabilities to provide the context if user is uploading those files in the chatbot unless I integrate it with another Azure AI services like Document Intelligence?