Golang + postgres storing gob data










0















I'm trying to store encoded data using encoding/gob from Golang into Postgres. I'm using Gorm as well.



First, sending form data using



if err := json.NewDecoder(r.Body).Decode(model); err != nil 
http.Error(w, err.Error(), http.StatusBadRequest)
return



Currently client_encoding is set to UTF8 in the postgres database. Here's what I'm using to encode:



type payload struct 
Key byte
Nonce *[nonceLength]byte
Message byte


// Encrypt message
p.Message = secretbox.Seal(p.Message, plaintext, p.Nonce, key) // key set prior to this

buf := &bytes.Buffer
if err := gob.NewEncoder(buf).Encode(p); err != nil
return nil, err


return buf.Bytes(), nil


Then I store string(buf.Bytes()) which is stored in the database column which is currently a string type. Now I'm a novice with encoding, and I think gob just has a different encoding for my database. I'm receiving this error in console:



(pq: invalid byte sequence for encoding "UTF8": 0xff)



I've been following this gist for encryption/decryption:
https://gist.github.com/fuzzyami/f3a7231037166117a6fef9607960aee7



From what I've read, I shouldn't be encoding structs into the db, in this case p, unless using gob. Correct me if I'm wrong with that (can't find the resource at the moment where I found this).



Is anyone able to point me in the right direction for storing this data in Postgres which is decrypted later? I'm clearly not understanding the encoding process, and not entirely sure where to start out here with reading resources, so any help is appreciated!










share|improve this question




























    0















    I'm trying to store encoded data using encoding/gob from Golang into Postgres. I'm using Gorm as well.



    First, sending form data using



    if err := json.NewDecoder(r.Body).Decode(model); err != nil 
    http.Error(w, err.Error(), http.StatusBadRequest)
    return



    Currently client_encoding is set to UTF8 in the postgres database. Here's what I'm using to encode:



    type payload struct 
    Key byte
    Nonce *[nonceLength]byte
    Message byte


    // Encrypt message
    p.Message = secretbox.Seal(p.Message, plaintext, p.Nonce, key) // key set prior to this

    buf := &bytes.Buffer
    if err := gob.NewEncoder(buf).Encode(p); err != nil
    return nil, err


    return buf.Bytes(), nil


    Then I store string(buf.Bytes()) which is stored in the database column which is currently a string type. Now I'm a novice with encoding, and I think gob just has a different encoding for my database. I'm receiving this error in console:



    (pq: invalid byte sequence for encoding "UTF8": 0xff)



    I've been following this gist for encryption/decryption:
    https://gist.github.com/fuzzyami/f3a7231037166117a6fef9607960aee7



    From what I've read, I shouldn't be encoding structs into the db, in this case p, unless using gob. Correct me if I'm wrong with that (can't find the resource at the moment where I found this).



    Is anyone able to point me in the right direction for storing this data in Postgres which is decrypted later? I'm clearly not understanding the encoding process, and not entirely sure where to start out here with reading resources, so any help is appreciated!










    share|improve this question


























      0












      0








      0








      I'm trying to store encoded data using encoding/gob from Golang into Postgres. I'm using Gorm as well.



      First, sending form data using



      if err := json.NewDecoder(r.Body).Decode(model); err != nil 
      http.Error(w, err.Error(), http.StatusBadRequest)
      return



      Currently client_encoding is set to UTF8 in the postgres database. Here's what I'm using to encode:



      type payload struct 
      Key byte
      Nonce *[nonceLength]byte
      Message byte


      // Encrypt message
      p.Message = secretbox.Seal(p.Message, plaintext, p.Nonce, key) // key set prior to this

      buf := &bytes.Buffer
      if err := gob.NewEncoder(buf).Encode(p); err != nil
      return nil, err


      return buf.Bytes(), nil


      Then I store string(buf.Bytes()) which is stored in the database column which is currently a string type. Now I'm a novice with encoding, and I think gob just has a different encoding for my database. I'm receiving this error in console:



      (pq: invalid byte sequence for encoding "UTF8": 0xff)



      I've been following this gist for encryption/decryption:
      https://gist.github.com/fuzzyami/f3a7231037166117a6fef9607960aee7



      From what I've read, I shouldn't be encoding structs into the db, in this case p, unless using gob. Correct me if I'm wrong with that (can't find the resource at the moment where I found this).



      Is anyone able to point me in the right direction for storing this data in Postgres which is decrypted later? I'm clearly not understanding the encoding process, and not entirely sure where to start out here with reading resources, so any help is appreciated!










      share|improve this question
















      I'm trying to store encoded data using encoding/gob from Golang into Postgres. I'm using Gorm as well.



      First, sending form data using



      if err := json.NewDecoder(r.Body).Decode(model); err != nil 
      http.Error(w, err.Error(), http.StatusBadRequest)
      return



      Currently client_encoding is set to UTF8 in the postgres database. Here's what I'm using to encode:



      type payload struct 
      Key byte
      Nonce *[nonceLength]byte
      Message byte


      // Encrypt message
      p.Message = secretbox.Seal(p.Message, plaintext, p.Nonce, key) // key set prior to this

      buf := &bytes.Buffer
      if err := gob.NewEncoder(buf).Encode(p); err != nil
      return nil, err


      return buf.Bytes(), nil


      Then I store string(buf.Bytes()) which is stored in the database column which is currently a string type. Now I'm a novice with encoding, and I think gob just has a different encoding for my database. I'm receiving this error in console:



      (pq: invalid byte sequence for encoding "UTF8": 0xff)



      I've been following this gist for encryption/decryption:
      https://gist.github.com/fuzzyami/f3a7231037166117a6fef9607960aee7



      From what I've read, I shouldn't be encoding structs into the db, in this case p, unless using gob. Correct me if I'm wrong with that (can't find the resource at the moment where I found this).



      Is anyone able to point me in the right direction for storing this data in Postgres which is decrypted later? I'm clearly not understanding the encoding process, and not entirely sure where to start out here with reading resources, so any help is appreciated!







      postgresql go gob






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 11 '18 at 18:43







      Greg Miller

















      asked Nov 11 '18 at 17:58









      Greg MillerGreg Miller

      147110




      147110






















          2 Answers
          2






          active

          oldest

          votes


















          0














          Use a bytea column in postgresql to store a byte, and skip the conversion to string.






          share|improve this answer























          • I've just tried to implement this, but when coding the body into the struct I get json: cannot unmarshal string into Go struct field Table.Column of type pq.ByteaArray. Any thoughts on getting around this? Seems like this can get a bit messy.

            – Greg Miller
            Nov 11 '18 at 18:35











          • That's coming from code that isn't in your question, and you're doing something very wrong.

            – hobbs
            Nov 11 '18 at 18:38











          • Just passing form data and decoding into a struct. I'll update the question now.

            – Greg Miller
            Nov 11 '18 at 18:42


















          0














          Took a look at https://golang.org/src/encoding/base64/example_test.go



          Was able to use



          return base64.StdEncoding.EncodeToString(buf.Bytes()), nil


          Which successfully stored in the database.






          share|improve this answer






















            Your Answer






            StackExchange.ifUsing("editor", function ()
            StackExchange.using("externalEditor", function ()
            StackExchange.using("snippets", function ()
            StackExchange.snippets.init();
            );
            );
            , "code-snippets");

            StackExchange.ready(function()
            var channelOptions =
            tags: "".split(" "),
            id: "1"
            ;
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function()
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled)
            StackExchange.using("snippets", function()
            createEditor();
            );

            else
            createEditor();

            );

            function createEditor()
            StackExchange.prepareEditor(
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader:
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            ,
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            );



            );













            draft saved

            draft discarded


















            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53251572%2fgolang-postgres-storing-gob-data%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown

























            2 Answers
            2






            active

            oldest

            votes








            2 Answers
            2






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            0














            Use a bytea column in postgresql to store a byte, and skip the conversion to string.






            share|improve this answer























            • I've just tried to implement this, but when coding the body into the struct I get json: cannot unmarshal string into Go struct field Table.Column of type pq.ByteaArray. Any thoughts on getting around this? Seems like this can get a bit messy.

              – Greg Miller
              Nov 11 '18 at 18:35











            • That's coming from code that isn't in your question, and you're doing something very wrong.

              – hobbs
              Nov 11 '18 at 18:38











            • Just passing form data and decoding into a struct. I'll update the question now.

              – Greg Miller
              Nov 11 '18 at 18:42















            0














            Use a bytea column in postgresql to store a byte, and skip the conversion to string.






            share|improve this answer























            • I've just tried to implement this, but when coding the body into the struct I get json: cannot unmarshal string into Go struct field Table.Column of type pq.ByteaArray. Any thoughts on getting around this? Seems like this can get a bit messy.

              – Greg Miller
              Nov 11 '18 at 18:35











            • That's coming from code that isn't in your question, and you're doing something very wrong.

              – hobbs
              Nov 11 '18 at 18:38











            • Just passing form data and decoding into a struct. I'll update the question now.

              – Greg Miller
              Nov 11 '18 at 18:42













            0












            0








            0







            Use a bytea column in postgresql to store a byte, and skip the conversion to string.






            share|improve this answer













            Use a bytea column in postgresql to store a byte, and skip the conversion to string.







            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Nov 11 '18 at 18:01









            hobbshobbs

            142k14150236




            142k14150236












            • I've just tried to implement this, but when coding the body into the struct I get json: cannot unmarshal string into Go struct field Table.Column of type pq.ByteaArray. Any thoughts on getting around this? Seems like this can get a bit messy.

              – Greg Miller
              Nov 11 '18 at 18:35











            • That's coming from code that isn't in your question, and you're doing something very wrong.

              – hobbs
              Nov 11 '18 at 18:38











            • Just passing form data and decoding into a struct. I'll update the question now.

              – Greg Miller
              Nov 11 '18 at 18:42

















            • I've just tried to implement this, but when coding the body into the struct I get json: cannot unmarshal string into Go struct field Table.Column of type pq.ByteaArray. Any thoughts on getting around this? Seems like this can get a bit messy.

              – Greg Miller
              Nov 11 '18 at 18:35











            • That's coming from code that isn't in your question, and you're doing something very wrong.

              – hobbs
              Nov 11 '18 at 18:38











            • Just passing form data and decoding into a struct. I'll update the question now.

              – Greg Miller
              Nov 11 '18 at 18:42
















            I've just tried to implement this, but when coding the body into the struct I get json: cannot unmarshal string into Go struct field Table.Column of type pq.ByteaArray. Any thoughts on getting around this? Seems like this can get a bit messy.

            – Greg Miller
            Nov 11 '18 at 18:35





            I've just tried to implement this, but when coding the body into the struct I get json: cannot unmarshal string into Go struct field Table.Column of type pq.ByteaArray. Any thoughts on getting around this? Seems like this can get a bit messy.

            – Greg Miller
            Nov 11 '18 at 18:35













            That's coming from code that isn't in your question, and you're doing something very wrong.

            – hobbs
            Nov 11 '18 at 18:38





            That's coming from code that isn't in your question, and you're doing something very wrong.

            – hobbs
            Nov 11 '18 at 18:38













            Just passing form data and decoding into a struct. I'll update the question now.

            – Greg Miller
            Nov 11 '18 at 18:42





            Just passing form data and decoding into a struct. I'll update the question now.

            – Greg Miller
            Nov 11 '18 at 18:42













            0














            Took a look at https://golang.org/src/encoding/base64/example_test.go



            Was able to use



            return base64.StdEncoding.EncodeToString(buf.Bytes()), nil


            Which successfully stored in the database.






            share|improve this answer



























              0














              Took a look at https://golang.org/src/encoding/base64/example_test.go



              Was able to use



              return base64.StdEncoding.EncodeToString(buf.Bytes()), nil


              Which successfully stored in the database.






              share|improve this answer

























                0












                0








                0







                Took a look at https://golang.org/src/encoding/base64/example_test.go



                Was able to use



                return base64.StdEncoding.EncodeToString(buf.Bytes()), nil


                Which successfully stored in the database.






                share|improve this answer













                Took a look at https://golang.org/src/encoding/base64/example_test.go



                Was able to use



                return base64.StdEncoding.EncodeToString(buf.Bytes()), nil


                Which successfully stored in the database.







                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Nov 11 '18 at 20:58









                Greg MillerGreg Miller

                147110




                147110



























                    draft saved

                    draft discarded
















































                    Thanks for contributing an answer to Stack Overflow!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid


                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.

                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function ()
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53251572%2fgolang-postgres-storing-gob-data%23new-answer', 'question_page');

                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    Popular posts from this blog

                    𛂒𛀶,𛀽𛀑𛂀𛃧𛂓𛀙𛃆𛃑𛃷𛂟𛁡𛀢𛀟𛁤𛂽𛁕𛁪𛂟𛂯,𛁞𛂧𛀴𛁄𛁠𛁼𛂿𛀤 𛂘,𛁺𛂾𛃭𛃭𛃵𛀺,𛂣𛃍𛂖𛃶 𛀸𛃀𛂖𛁶𛁏𛁚 𛂢𛂞 𛁰𛂆𛀔,𛁸𛀽𛁓𛃋𛂇𛃧𛀧𛃣𛂐𛃇,𛂂𛃻𛃲𛁬𛃞𛀧𛃃𛀅 𛂭𛁠𛁡𛃇𛀷𛃓𛁥,𛁙𛁘𛁞𛃸𛁸𛃣𛁜,𛂛,𛃿,𛁯𛂘𛂌𛃛𛁱𛃌𛂈𛂇 𛁊𛃲,𛀕𛃴𛀜 𛀶𛂆𛀶𛃟𛂉𛀣,𛂐𛁞𛁾 𛁷𛂑𛁳𛂯𛀬𛃅,𛃶𛁼

                    How do I collapse sections of code in Visual Studio Code for Windows?

                    ャフサォクコ ケウ,コ,ワ メ,ロスョノ゙,クネ,フムカヤヲニ,エコ゚ツ ウイオン゙ケワサネォキモュキォウイノンコチ゚メヌナイゥフュ,カヒウネェ ネ,ホノケ,ムュキ ッボーミュハ,チ ツス ィ メウイマヤ,゙ウチ ヅ ロ,ォジヌェ ャヌット ェ,マャ,チナエヒネソキツテ トホヲヲミーァ