Share via

How to decompress the GZIP file to json format

  • Article

Question

Wednesday, June 26, 2019 5:46 AM

Hi Team,

Have a one sql table with 1000's of records. My application team storing the xml file in the form of ZGIP in sql table. Is their any option to decompress the ZGIP to json format.

Want to decompress the data and store data into local system.

Thanks Bala Narasimha

All replies (18)

Friday, June 28, 2019 2:06 AM âś…Answered

Hi BaluChalla,

Thanks for understanding.

According to your requirements, I make a simple code to read data from database and write it's text to a  xml file.

Th following code has some key comments, you could have a look.

static void Main(string[] args)
        {
            string fileName = @"D:\Gzip\t.xml";
            byte[] file = File.ReadAllBytes(fileName);
            byte[] compress = Compress(file);
            string con = @"";
            SqlConnection connection = new SqlConnection(con);
            connection.Open();
            string sql = "insert into Gzip (Id,Name,requestXML,responseXML) values(@Id,@Name,@requestXML,@responseXML)";
            SqlCommand cmd = new SqlCommand(sql, connection);
            SqlParameter param = cmd.Parameters.Add("@Id", SqlDbType.Int);
            param.Value = 1;
            SqlParameter param1 = cmd.Parameters.Add("@Name", SqlDbType.NVarChar);
            param1.Value = "test1";
            SqlParameter param2 = cmd.Parameters.Add("@requestXML", SqlDbType.VarBinary);
            param2.Value = file;
            SqlParameter param3 = cmd.Parameters.Add("@responseXML", SqlDbType.VarBinary);
            param3.Value = compress;
            cmd.ExecuteNonQuery();                      // insert byte data to database
            connection.Close();
            Console.WriteLine("success");                  
            connection.Open();
            string newsql = "select * from Gzip";
            cmd = new SqlCommand(newsql,connection);
            SqlDataReader reader = cmd.ExecuteReader();
            int i = 1;
            while(reader.Read())
            {
                var m = (byte[])reader[3];                                // read responseXML from database
                string text = Unzip(m);                                   // convert byte array to string
                File.WriteAllText("test" + i.ToString() + ".xml", text);
            }
            Console.WriteLine("ok");
            connection.Close();
            Console.ReadKey();

        }

        public static byte[] Compress(byte[] data)
        {
            using (var compressedStream = new MemoryStream())
            using (var zipStream = new GZipStream(compressedStream, CompressionMode.Compress))
            {
                zipStream.Write(data, 0, data.Length);
                zipStream.Close();
                return compressedStream.ToArray();
            }
        }

        public static void CopyTo(Stream src, Stream dest)
        {
            byte[] bytes = new byte[4096];
            int cnt;
            while ((cnt = src.Read(bytes, 0, bytes.Length)) != 0)
            {
                dest.Write(bytes, 0, cnt);
            }
        }

        public static string Unzip(byte[] bytes)
        {
            using (var msi = new MemoryStream(bytes))
            using (var mso = new MemoryStream())
            {
                using (var gs = new GZipStream(msi, CompressionMode.Decompress))
                {
                    //gs.CopyTo(mso);
                    CopyTo(gs, mso);
                }


                return Encoding.UTF8.GetString(mso.ToArray());
            }
        }

Result:

Best Regards,

Jack

MSDN Community Support
Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact [email protected].

Wednesday, June 26, 2019 8:45 AM

Hi Team,

Have a one sql table with 1000's of records. My application team storing the xml file in the form of ZGIP in sql table. Is their any option to decompress the ZGIP to json format.

Want to decompress the data and store data into local system.

Thanks Bala Narasimha

IMO, the best you can do is decompress the data,  write a program that is going to convert the XML data over to a Json format and do that.  

Wednesday, June 26, 2019 9:28 AM

Hi,

Thanks for your response.

Am new to C# could you please help on this.

Thanks Bala Narasimha

Wednesday, June 26, 2019 9:39 AM

Hi,

Thanks for your response.

Am new to C# could you please help on this.

Thanks Bala Narasimha

I don't think you can load all the data into an XML document, but rather, load segments of the XML file into an XML document.

You can look at what others have done in similar satiations for an answer on how to do it.

https://www.google.com/search?source=hp&ei=ADwTXaekHtSktQaJtqDIBw&q=convert+xml+to+json+c%23&oq=convert+xml+to+json&gs_l=psy-ab.1.0.35i39j0l9.2729.13994..16451...1.0..0.394.2338.14j5j0j1......0....1..gws-wiz.....6..0i131j0i20i263.-86SiF7VNg4

 

Thursday, June 27, 2019 2:20 AM

Hi BaluChalla,

Thank you for posting here.

Based on your description, you want to decompress the GZIP file to json format.

First, we need to decompress the GZIP file to the xml file.

Then, we could convert xml to json string.

Finally, we write the json string to json file.

Here is a simple code for your reference.

 private static string directoryPath = @"D:\Gzip";
        static void Main(string[] args)
        {
            
            DirectoryInfo directorySelected = new DirectoryInfo(directoryPath);
            string filename = string.Empty;
            int count = 1;
            foreach (FileInfo fileToDecompress in directorySelected.GetFiles("*.gz"))
            {
                Decompress(fileToDecompress,out filename);
                XmlDocument doc = new XmlDocument();
                doc.Load(filename);
                string jsonText = JsonConvert.SerializeXmlNode(doc);
                File.WriteAllText(Path.Combine(directoryPath, count.ToString()+".json"), jsonText);
            }
        }
      
        public static void Decompress(FileInfo fileToDecompress,out string newFileName)
        {
            using (FileStream originalFileStream = fileToDecompress.OpenRead())
            {
                string currentFileName = fileToDecompress.FullName;
                  newFileName = currentFileName.Remove(currentFileName.Length - fileToDecompress.Extension.Length);

                using (FileStream decompressedFileStream = File.Create(newFileName))
                {
                    using (GZipStream decompressionStream = new GZipStream(originalFileStream, CompressionMode.Decompress))
                    {
                        decompressionStream.CopyTo(decompressedFileStream);
                        Console.WriteLine($"Decompressed: {fileToDecompress.FullName}");
                    }
                }
            }
        }

Result:

Best Regards,

Jack

MSDN Community Support
Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact [email protected].

Thursday, June 27, 2019 4:50 AM

Hi Jack,

Thanks for your response.

here my source is sql table. How can i get data from sql table and unzip data. Could you please help on this.

Thanks Bala Narasimha

Thursday, June 27, 2019 5:18 AM

Hi BaluChalla,

Thanks for the feedback.

I want to know what your database design is. Could you tell me how do you store it? If you provide the related code, which will help us to solve the problem.

Best Regards,

Jack

MSDN Community Support
Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact [email protected].

Thursday, June 27, 2019 5:29 AM

Hi Jack,

Thanks for update.

1. My application team data storing in the format of ZIP in sql table.
2. I need to convert that data into text and again store in a database

sample data:

ColumnName Value
profilerId 1
requestXML 0x1F8B0800000000000400858FC14EC3301044EF48FC83E57BB3495BF550392EA8D7362AB4485C5D67492C1A1B7937753F9F104124B870DC796FB41AB5B9751771C5482EF85216592E057A1B6AE79B52F6FC362B5672A3EFEFD411E9CBD946348CCF4F62E8795ADFA82E65CBFCB106482965699185D8C03CCF0B78DDEF8EB6C5CEC84976FFCB33E7898DB72887AF42A8476B43EFB9EABB3346BDDF5683BDCC570A7E83D17D218C95E950BF9BABE3D6289892911F0C510AB1FEE10FC57CA1604A47E7646283AC4F48ACE0FB18E6C39FFDFA1399270E463A010000
responseXML 0x1F8B08000000000004008590B18EC2301044FB93EE1F2CF78E6D12E5129498E27405123484E25A6BB30991C041F6E6C2E71310208E867276DEEC48532C4E873DFB431FBADE955C478A3374D0D79D6B4B3E502374CA17E6F3A3A8305C986F8F967053B129E7C2FC14EA92EF888E7329C7718CC638EA7D2B674A69F9BB5E55B0C383E50FB87B0F8BCE05B20E904FAD8CDD7B2BB23404437EC042FEBF3D63CBDA3439400E0842CFE25424598A224B621099822657FA0B92A479BC98F86B7C6B7D8B64B618A8903771357EBCEF7D60F232817CD9C09C014D29514A3E010000
createdOn 6/1/2015 11:33

Thanks Bala Narasimha

Thursday, June 27, 2019 5:38 AM

Hi BaluChalla,

I want to know what the requestXML and resonseXML are. Is there a difference between them?

I guess that the requestXML and responseXML are binary types, is right?

Best Regards,

Jack

MSDN Community Support
Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact [email protected].

Thursday, June 27, 2019 5:51 AM

Hi Jack,

requestXML and responseXML - basically these two field contain a xml data but while storing in database we are converting to zip format and storing in database.

Now i want to extract that data to original format.

Thanks Bala Narasimha

My application team using bellow code to unzipping code.

public static void CopyTo(Stream src, Stream dest)
        {
            byte[] bytes = new byte[4096];

            int cnt;

            while ((cnt = src.Read(bytes, 0, bytes.Length)) != 0)
            {
                dest.Write(bytes, 0, cnt);
            }
        }

public static string Unzip(byte[] bytes)
        {
            using (var msi = new MemoryStream(bytes))
            using (var mso = new MemoryStream())
            {
                using (var gs = new GZipStream(msi, CompressionMode.Decompress))
                {
                    //gs.CopyTo(mso);
                    CopyTo(gs, mso);
                }

                return Encoding.UTF8.GetString(mso.ToArray());
            }
        }

Could you please help on this.

Thursday, June 27, 2019 5:55 AM

Hi BaluChalla,

Thanks for the feedback.

I have to ask if it is related to your initial problem that decompress the ZGIP to json format. why not mention the json in the last reply?

Best Regards,

Jack

MSDN Community Support
Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact [email protected].

Thursday, June 27, 2019 6:56 AM

Hi Jack,

Thanks for feedback.

Initially i Want to convert zip data into text format 

Thanks Bala Narasimha

Thursday, June 27, 2019 7:25 AM

Hi BaluChalla,

Thanks for the feedback.

I have tested the code, it could convert zip data into text format successfully.

Code:

static void Main(string[] args)
        {
            string fileName = @"D:\Gzip\t.xml";
            byte[] file = File.ReadAllBytes(fileName);
            byte[] compress = Compress(file);
            string t = Unzip(compress);
        }

        public static byte[] Compress(byte[] data)
        {
            using (var compressedStream = new MemoryStream())
            using (var zipStream = new GZipStream(compressedStream, CompressionMode.Compress))
            {
                zipStream.Write(data, 0, data.Length);
                zipStream.Close();
                return compressedStream.ToArray();
            }
        }

        public static void CopyTo(Stream src, Stream dest)
        {
            byte[] bytes = new byte[4096];
            int cnt;
            while ((cnt = src.Read(bytes, 0, bytes.Length)) != 0)
            {
                dest.Write(bytes, 0, cnt);
            }
        }

        public static string Unzip(byte[] bytes)
        {
            using (var msi = new MemoryStream(bytes))
            using (var mso = new MemoryStream())
            {
                using (var gs = new GZipStream(msi, CompressionMode.Decompress))
                {
                    //gs.CopyTo(mso);
                    CopyTo(gs, mso);
                }


                return Encoding.UTF8.GetString(mso.ToArray());
            }
        }

Result:

If you have any other question, please feel free to let us know.

Best Regards,

Jack

MSDN Community Support
Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact [email protected].

Thursday, June 27, 2019 9:31 AM

Hi Jack,

Thanks for your support.

Based on your guidance have modified given code and am storing unzip files into local system. Instead of  storing local system i want to store data into MongoDB. 

Could you please help on same. 

Please find-out my code:

#region Namespaces
using System;
using System.Data.SqlClient;
using System.IO;
using System.IO.Compression;
using System.Text;
#endregion

namespace UnZipData
{

    class Program
    {
        static void Main(string[] args)
        {
            SqlConnection con;

            SqlDataReader reader;

            try

            {
                string connection = System.Configuration.ConfigurationManager.ConnectionStrings["Profiler"].ConnectionString;

                con = new SqlConnection(connection);
                con.Open();

                reader = new SqlCommand("SELECT TOP 100 Id,Name,requestXML,responseXML FROM TableName order by 1 desc", con).ExecuteReader();

                if (reader.HasRows)
                {
                    while (reader.Read())
                    {
                        // WriteAllText creates a file, writes the specified string to the file,
                        // and then closes the file.    You do NOT need to call Flush() or Close().

                        string filePath = $@"D:\FareRules\ +  reader.GetInt32(0);
                        File.WriteAllText(filePath, Unzip((byte[])reader["requestXML"]));
                    }
                }
                else
                {
                    Console.WriteLine("No rows found.");
                }
                reader.Close();
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
            }
        }

        public static void CopyTo(Stream src, Stream dest)
        {
            byte[] bytes = new byte[4096];

            int cnt;

            while ((cnt = src.Read(bytes, 0, bytes.Length)) != 0)
            {
                dest.Write(bytes, 0, cnt);
            }
        }

        public static string Unzip(byte[] bytes)
        {
            using (var msi = new MemoryStream(bytes))
            using (var mso = new MemoryStream())
            {
                using (var gs = new GZipStream(msi, CompressionMode.Decompress))
                {
                    //gs.CopyTo(mso);
                    CopyTo(gs, mso);
                }

                return Encoding.UTF8.GetString(mso.ToArray());
            }
        }
    }
}

Thanks Bala Narasimha

Thursday, June 27, 2019 9:48 AM

Hi BaluChalla,

Thanks for the feedback.

If you used the MongoDB, I suggest that you could post it in the third-party forum. Because it is a third-party product. Our forum doesn't support third-party product.

The Visual C# forum discusses and asks questions about the C# programming language, IDE, libraries, samples, and tools.

**Note:**This response contains a reference to a third party World Wide Web site. Microsoft is providing this information as a convenience to you. Microsoft does not control these sites and has not tested any software or information found on these sites; Therefore, Microsoft cannot make any representations regarding the quality, safety, or suitability of any software or information found there. There are inherent dangers in the use of any software found on the Internet, and Microsoft cautions you to make sure that you completely understand the risk before retrieving any software from the Internet.

Best Regards,

Jack

MSDN Community Support
Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact [email protected].

Thursday, June 27, 2019 10:05 AM

Hi Jack,

Thanks for your support.

It is possible to store same data into SQL database, if it is possible please help on same.

Thanks Bala Narasimha

Tuesday, July 2, 2019 4:46 AM

Hi Jack,

Thanks for your support.

It is possible to store extracted xml data into sql database. if it is possible please help on same.

Thanks Bala Narasimha

Tuesday, July 2, 2019 5:18 AM

Hi BaluChalla,

Thanks for the feedback.

>>It is possible to store extracted xml data into sql database.

I have done it already. Therefore, I think you could mark it as an answer. If you have new questions, you could create a new thread in C# forum.

Best Regards,

Jack

MSDN Community Support
Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact [email protected].