Kafka JDBC Source Connector is not reading data from mysql with sql query?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;
0
I have created a local database for testing purpose on MySQL. When I'm trying to read the data from the database, it's reading fine. When I use the SQL query in connector configuration, it's just running but not reading any data from the database. Any idea what might cause this?
Here is my connector config:
name=just-like
acks=0
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
key.converter=io.confluent.connect.avro.AvroConverter
key.converter.schema.registry.url=http://localhost:8081
value.converter=io.confluent.connect.avro.AvroConverter
value.converter.schema.registry.url=http://localhost:8081
connection.url=jdbc:mysql://localhost:3306/DBtest?user=**&password=**
connection.user=**
connection.password=**
table.whitelist=test
mode=timestamp
timestamp.column.name=createddate
query= select column_name from table_name
validate.non.null=False
topic.prefix=sql-
Database Schema
mysql jdbc apache-kafka apache-kafka-connect confluent
asked Nov 14 '18 at 9:25
DP0808DP0808
13
|
show 2 more comments
0
I have created a local database for testing purpose on MySQL. When I'm trying to read the data from the database, it's reading fine. When I use the SQL query in connector configuration, it's just running but not reading any data from the database. Any idea what might cause this?
Here is my connector config:
name=just-like
acks=0
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
key.converter=io.confluent.connect.avro.AvroConverter
key.converter.schema.registry.url=http://localhost:8081
value.converter=io.confluent.connect.avro.AvroConverter
value.converter.schema.registry.url=http://localhost:8081
connection.url=jdbc:mysql://localhost:3306/DBtest?user=**&password=**
connection.user=**
connection.password=**
table.whitelist=test
mode=timestamp
timestamp.column.name=createddate
query= select column_name from table_name
validate.non.null=False
topic.prefix=sql-
Database Schema
mysql jdbc apache-kafka apache-kafka-connect confluent
asked Nov 14 '18 at 9:25
DP0808DP0808
13
Can you show your table schema and current data in the table? Please create a Minimal, Complete, and Verifiable example
– cricket_007
Nov 14 '18 at 15:33
You may want to review your configuration, because as posted it seems to be invalid (multiple properties on a single line), which might well be the cause of your problem (but given you botched the initial formatting of your post, I can't be sure).
– Mark Rotteveel
Nov 14 '18 at 16:24
Please have a look on question again. i have added required details.
– DP0808
Nov 15 '18 at 8:35
Rather than just"select column_name from table_name", can you please show the actual query? Andtable.whitelist=testwouldn't read fromtable_name, so that seems wrong. And thenlocalhost:3306/DBtest user=**&password=**has a space in it, so I don't think that is correct
– cricket_007
Nov 17 '18 at 0:36
I have tried as you suggested but as soon as i put sql command it stopped working and when i remove query it's working fine and pulling data but it's fetching all columns from a table and i want to access few. Please let me know.
– DP0808
Nov 19 '18 at 10:02
|
show 2 more comments
0
0
0
I have created a local database for testing purpose on MySQL. When I'm trying to read the data from the database, it's reading fine. When I use the SQL query in connector configuration, it's just running but not reading any data from the database. Any idea what might cause this?
Here is my connector config:
name=just-like
acks=0
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
key.converter=io.confluent.connect.avro.AvroConverter
key.converter.schema.registry.url=http://localhost:8081
value.converter=io.confluent.connect.avro.AvroConverter
value.converter.schema.registry.url=http://localhost:8081
connection.url=jdbc:mysql://localhost:3306/DBtest?user=**&password=**
connection.user=**
connection.password=**
table.whitelist=test
mode=timestamp
timestamp.column.name=createddate
query= select column_name from table_name
validate.non.null=False
topic.prefix=sql-
Database Schema
mysql jdbc apache-kafka apache-kafka-connect confluent
asked Nov 14 '18 at 9:25
DP0808DP0808
13
I have created a local database for testing purpose on MySQL. When I'm trying to read the data from the database, it's reading fine. When I use the SQL query in connector configuration, it's just running but not reading any data from the database. Any idea what might cause this?
Here is my connector config:
name=just-like
acks=0
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
key.converter=io.confluent.connect.avro.AvroConverter
key.converter.schema.registry.url=http://localhost:8081
value.converter=io.confluent.connect.avro.AvroConverter
value.converter.schema.registry.url=http://localhost:8081
connection.url=jdbc:mysql://localhost:3306/DBtest?user=**&password=**
connection.user=**
connection.password=**
table.whitelist=test
mode=timestamp
timestamp.column.name=createddate
query= select column_name from table_name
validate.non.null=False
topic.prefix=sql-
Database Schema
mysql jdbc apache-kafka apache-kafka-connect confluent
mysql jdbc apache-kafka apache-kafka-connect confluent
asked Nov 14 '18 at 9:25
DP0808DP0808
13
asked Nov 14 '18 at 9:25
DP0808DP0808
13
edited Nov 19 '18 at 6:11
DP0808
asked Nov 14 '18 at 9:25
DP0808DP0808
13
asked Nov 14 '18 at 9:25
DP0808DP0808
13
asked Nov 14 '18 at 9:25
DP0808DP0808
13
13
Can you show your table schema and current data in the table? Please create a Minimal, Complete, and Verifiable example
– cricket_007
Nov 14 '18 at 15:33
You may want to review your configuration, because as posted it seems to be invalid (multiple properties on a single line), which might well be the cause of your problem (but given you botched the initial formatting of your post, I can't be sure).
– Mark Rotteveel
Nov 14 '18 at 16:24
Please have a look on question again. i have added required details.
– DP0808
Nov 15 '18 at 8:35
Rather than just"select column_name from table_name", can you please show the actual query? Andtable.whitelist=testwouldn't read fromtable_name, so that seems wrong. And thenlocalhost:3306/DBtest user=**&password=**has a space in it, so I don't think that is correct
– cricket_007
Nov 17 '18 at 0:36
I have tried as you suggested but as soon as i put sql command it stopped working and when i remove query it's working fine and pulling data but it's fetching all columns from a table and i want to access few. Please let me know.
– DP0808
Nov 19 '18 at 10:02
|
show 2 more comments
Can you show your table schema and current data in the table? Please create a Minimal, Complete, and Verifiable example
– cricket_007
Nov 14 '18 at 15:33
You may want to review your configuration, because as posted it seems to be invalid (multiple properties on a single line), which might well be the cause of your problem (but given you botched the initial formatting of your post, I can't be sure).
– Mark Rotteveel
Nov 14 '18 at 16:24
Please have a look on question again. i have added required details.
– DP0808
Nov 15 '18 at 8:35
Rather than just"select column_name from table_name", can you please show the actual query? Andtable.whitelist=testwouldn't read fromtable_name, so that seems wrong. And thenlocalhost:3306/DBtest user=**&password=**has a space in it, so I don't think that is correct
– cricket_007
Nov 17 '18 at 0:36
I have tried as you suggested but as soon as i put sql command it stopped working and when i remove query it's working fine and pulling data but it's fetching all columns from a table and i want to access few. Please let me know.
– DP0808
Nov 19 '18 at 10:02
Can you show your table schema and current data in the table? Please create a Minimal, Complete, and Verifiable example
– cricket_007
Nov 14 '18 at 15:33
Can you show your table schema and current data in the table? Please create a Minimal, Complete, and Verifiable example
– cricket_007
Nov 14 '18 at 15:33
You may want to review your configuration, because as posted it seems to be invalid (multiple properties on a single line), which might well be the cause of your problem (but given you botched the initial formatting of your post, I can't be sure).
– Mark Rotteveel
Nov 14 '18 at 16:24
You may want to review your configuration, because as posted it seems to be invalid (multiple properties on a single line), which might well be the cause of your problem (but given you botched the initial formatting of your post, I can't be sure).
– Mark Rotteveel
Nov 14 '18 at 16:24
Please have a look on question again. i have added required details.
– DP0808
Nov 15 '18 at 8:35
Please have a look on question again. i have added required details.
– DP0808
Nov 15 '18 at 8:35
Rather than just
"select column_name from table_name", can you please show the actual query? And table.whitelist=test wouldn't read from table_name, so that seems wrong. And then localhost:3306/DBtest user=**&password=** has a space in it, so I don't think that is correct– cricket_007
Nov 17 '18 at 0:36
Rather than just
"select column_name from table_name", can you please show the actual query? And table.whitelist=test wouldn't read from table_name, so that seems wrong. And then localhost:3306/DBtest user=**&password=** has a space in it, so I don't think that is correct– cricket_007
Nov 17 '18 at 0:36
I have tried as you suggested but as soon as i put sql command it stopped working and when i remove query it's working fine and pulling data but it's fetching all columns from a table and i want to access few. Please let me know.
– DP0808
Nov 19 '18 at 10:02
I have tried as you suggested but as soon as i put sql command it stopped working and when i remove query it's working fine and pulling data but it's fetching all columns from a table and i want to access few. Please let me know.
– DP0808
Nov 19 '18 at 10:02
|
show 2 more comments
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53296784%2fkafka-jdbc-source-connector-is-not-reading-data-from-mysql-with-sql-query%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53296784%2fkafka-jdbc-source-connector-is-not-reading-data-from-mysql-with-sql-query%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Can you show your table schema and current data in the table? Please create a Minimal, Complete, and Verifiable example
– cricket_007
Nov 14 '18 at 15:33
You may want to review your configuration, because as posted it seems to be invalid (multiple properties on a single line), which might well be the cause of your problem (but given you botched the initial formatting of your post, I can't be sure).
– Mark Rotteveel
Nov 14 '18 at 16:24
Please have a look on question again. i have added required details.
– DP0808
Nov 15 '18 at 8:35
Rather than just
"select column_name from table_name", can you please show the actual query? Andtable.whitelist=testwouldn't read fromtable_name, so that seems wrong. And thenlocalhost:3306/DBtest user=**&password=**has a space in it, so I don't think that is correct– cricket_007
Nov 17 '18 at 0:36
I have tried as you suggested but as soon as i put sql command it stopped working and when i remove query it's working fine and pulling data but it's fetching all columns from a table and i want to access few. Please let me know.
– DP0808
Nov 19 '18 at 10:02