GPT-4o Finetuning Failed

Question

GPT-4o Finetuning Failed

Nas 0

I'm receiving an error.

After preparing data files following guidelines (https://learn.microsoft.com/en-us/azure/ai-services/openai/tutorials/fine-tune?tabs=command-line).

During Preprocessing Files:

status : Training file: Preprocessing Summary: The provided data failed validation due to: contains invalid schema (22). Please visit our docs to learn how to resolve these issues, and try again. Details - Samples of lines per error type: contains invalid schema: Line numbers --> 1, 3, 4, 5, 6, 7, 9, 11, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26

After: When I force my files to UTF-8 encoding without BOM, I now get error like this, just simple File Preprocessing failed.

File preprocessing failed

Any input will be appreciated!!.
Thanks.

Ravada Shivaprasad 10 Reputation points Microsoft External Staff

2025-04-22T13:27:00.1566667+00:00
Hi Nas
To resolve your GPT-4o model fine-tuning issues completely, let's address both the schema validation error and preprocessing failure systematically.

First, ensure your training file follows Microsoft's official JSON format requirements where each line must contain properly formatted prompt completion pairs with exact JSON syntax and also the file must be encoded in UTF-8 without BOM (Byte Order Mark) as explicitly stated as above.

To convert your file encoding appropriately, you can use PowerShell commands for to fix Encoding

Get-Content input.json | Set-Content -Encoding utf8 output.json

Each line should follow the exact format shown in Microsoft's examples and also verify that your file meets all validation criteria by checking for proper JSON structure, ensuring each line contains both prompt and completion fields, and confirming consistent line endings throughout the file. If you still encounter errors after implementing these changes, check for hidden characters at the beginning of your file and verify that all prompts and completions are properly quoted, as these are common issues documented in Microsoft's troubleshooting guide.

Reference - Fine Tuning Documentation ,Ai Docs

Thanks

1 answer

Your answer

Ravada Shivaprasad 10 Reputation points Microsoft External Staff

2025-04-22T13:27:00.1566667+00:00

Hi Nas
To resolve your GPT-4o model fine-tuning issues completely, let's address both the schema validation error and preprocessing failure systematically.

First, ensure your training file follows Microsoft's official JSON format requirements where each line must contain properly formatted prompt completion pairs with exact JSON syntax and also the file must be encoded in UTF-8 without BOM (Byte Order Mark) as explicitly stated as above.

To convert your file encoding appropriately, you can use PowerShell commands for to fix Encoding

Get-Content input.json | Set-Content -Encoding utf8 output.json

Each line should follow the exact format shown in Microsoft's examples and also verify that your file meets all validation criteria by checking for proper JSON structure, ensuring each line contains both prompt and completion fields, and confirming consistent line endings throughout the file. If you still encounter errors after implementing these changes, check for hidden characters at the beginning of your file and verify that all prompts and completions are properly quoted, as these are common issues documented in Microsoft's troubleshooting guide.

Reference - Fine Tuning Documentation ,Ai Docs

Thanks

Answer 1

Nas 0

Hey @Ravada Shivaprasad , Thank you for your prompt response. I still see the same error as can be seen in the images below. I validated the JSON schema in both the training and validation files, which was fine.

I noted that my JSON file contains a variety of characters, HTML tags, and Unicode escape sequences (like \ud83d\udc49). Will this affect the training?

User's image

Ravada Shivaprasad 10 Reputation points Microsoft External Staff

2025-04-24T09:19:03.61+00:00
Hi Nas

When working with JSON files containing Unicode characters such as ('\ud83d\udc49'), it's important to handle them properly to avoid issues during the fine-tuning process. These Unicode escape sequences are often used to represent emojis or other special characters in text data. While they appear as valid-looking strings in the raw JSON, they can cause problems when processed by some systems unless converted to their proper UTF-8 encoded representations.
For that you have to modify your data processing pipeline to convert these escape sequences into actual Unicode characters. You need to implement proper data preprocessing. Here's a comprehensive solution that handles both HTML tags and Unicode escape sequences

# Remove HTML tags while preserving content $htmlCleaned = (Get-Content "input.json" -Raw) | ForEach-Object { [regex]::Replace($_, '<.*?>', '') } $unicodeJson = $htmlCleaned | ForEach-Object { [regex]::Replace($_, '\\\\u([0-9a-fA-F]{4})', { param($match) $code = [Convert]::ToInt32($match.Groups[1].Value, 16) [char]$code }) } # Save the cleaned data with proper UTF-8 encoding $unicodeJson | Set-Content "cleaned_data.json" -Encoding utf8

This preprocessing approach ensures that your data is properly formatted for Azure OpenAI fine-tuning. The script first removes HTML tags while preserving the actual content, then converts Unicode escape sequences into their proper UTF-8 representations.
After applying this preprocessing need to validate your data to ensure it meets Azure OpenAI's requirements. By implementing this preprocessing pipeline, you can ensure that your training data is properly formatted and free of potential issues that could affect your model's performance. The cleaned data will be more reliable for training and will help prevent common errors related to character encoding and HTML tags.

Thanks

Share via

GPT-4o Finetuning Failed

1 answer

Your answer