Joe Julian
2016-Feb-01 16:46 UTC
[Gluster-users] glustershd fail to start after 3.7.7 upgrade
WTF? if (!xattrs_list) { ret = -EINVAL; gf_msg (this->name, GF_LOG_ERROR, -ret, AFR_MSG_NO_CHANGELOG, "Unable to fetch afr pending changelogs. Is op-version" " >= 30707?"); goto out; } This is not going to work. Look at the server upgrade process: Upgrade one side of the replica, wait for self-heals to finish, upgrade the other side. This breaks that process. On 02/01/2016 07:08 AM, Emmanuel Dreyfus wrote:> Hi > > After upgrading to 3.7.7, glustershd will not start anymore. Here are > the log. It seems that there a queued ops with odler version number > hat kill it. > > [2016-02-01 15:03:04.832962] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/pkg/sbin/glusterfs: Started running /usr/pkg/sbin/glusterfs version 3.7.7 (args: /usr/pkg/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p /var/lib/glusterd/glustershd/run/glustershd.pid -l /var/log/glusterfs/glustershd.log -S /var/run/gluster/c7e5574af4b5b4ffdcb61b1d5e63d8da.socket --xlator-option *replicate*.node-uuid=85eb78cd-8ffa-49ca-b3e7-d5030bc3124d) > [2016-02-01 15:03:05.065456] I [graph.c:269:gf_add_cmdline_options] 0-gfs-replicate-3: adding option 'node-uuid' for volume 'gfs-replicate-3' with value '85eb78cd-8ffa-49ca-b3e7-d5030bc3124d' > [2016-02-01 15:03:05.065510] I [graph.c:269:gf_add_cmdline_options] 0-gfs-replicate-2: adding option 'node-uuid' for volume 'gfs-replicate-2' with value '85eb78cd-8ffa-49ca-b3e7-d5030bc3124d' > [2016-02-01 15:03:05.065532] I [graph.c:269:gf_add_cmdline_options] 0-gfs-replicate-1: adding option 'node-uuid' for volume 'gfs-replicate-1' with value '85eb78cd-8ffa-49ca-b3e7-d5030bc3124d' > [2016-02-01 15:03:05.065552] I [graph.c:269:gf_add_cmdline_options] 0-gfs-replicate-0: adding option 'node-uuid' for volume 'gfs-replicate-0' with value '85eb78cd-8ffa-49ca-b3e7-d5030bc3124d' > [2016-02-01 15:03:05.065866] E [MSGID: 108040] [afr.c:418:init] 0-gfs-replicate-3: Unable to fetch afr pending changelogs. Is op-version >= 30707? [Invalid argument] > [2016-02-01 15:03:05.066002] E [MSGID: 101019] [xlator.c:433:xlator_init] 0-gfs-replicate-3: Initialization of volume 'gfs-replicate-3' failed, review your volfile again > [2016-02-01 15:03:05.066024] E [graph.c:322:glusterfs_graph_init] 0-gfs-replicate-3: initializing translator failed > [2016-02-01 15:03:05.066045] E [graph.c:661:glusterfs_graph_activate] 0-graph: init failed > [2016-02-01 15:03:05.066348] W [glusterfsd.c:1236:cleanup_and_exit] (-->0xbbbd9d64 <rpc_clnt_handle_reply+452> at /usr/pkg/lib/libgfrpc.so.0 -->0x8055f28 <mgmt_getspec_cbk+850> at /usr/pkg/sbin/glusterfs -->0x80518a4 <glusterfs_process_volfp+430> at /usr/pkg/sbin/glusterfs ) 0-: received signum (0), shutting down >
Pranith Kumar Karampuri
2016-Feb-02 03:24 UTC
[Gluster-users] glustershd fail to start after 3.7.7 upgrade
On 02/01/2016 10:16 PM, Joe Julian wrote:> WTF? > > if (!xattrs_list) { > ret = -EINVAL; > gf_msg (this->name, GF_LOG_ERROR, -ret, > AFR_MSG_NO_CHANGELOG, > "Unable to fetch afr pending changelogs. Is > op-version" > " >= 30707?"); > goto out; > } > > This is not going to work. Look at the server upgrade process: Upgrade > one side of the replica, wait for self-heals to finish, upgrade the > other side. This breaks that process.Sorry guys, this is my screw up, didn't think of this case in the design/review. I will fix this immediately. I hope the rpms are not yet available for 3.7.7? Pranith> > On 02/01/2016 07:08 AM, Emmanuel Dreyfus wrote: >> Hi >> >> After upgrading to 3.7.7, glustershd will not start anymore. Here are >> the log. It seems that there a queued ops with odler version number >> hat kill it. >> >> [2016-02-01 15:03:04.832962] I [MSGID: 100030] >> [glusterfsd.c:2318:main] 0-/usr/pkg/sbin/glusterfs: Started running >> /usr/pkg/sbin/glusterfs version 3.7.7 (args: /usr/pkg/sbin/glusterfs >> -s localhost --volfile-id gluster/glustershd -p >> /var/lib/glusterd/glustershd/run/glustershd.pid -l >> /var/log/glusterfs/glustershd.log -S >> /var/run/gluster/c7e5574af4b5b4ffdcb61b1d5e63d8da.socket >> --xlator-option >> *replicate*.node-uuid=85eb78cd-8ffa-49ca-b3e7-d5030bc3124d) >> [2016-02-01 15:03:05.065456] I [graph.c:269:gf_add_cmdline_options] >> 0-gfs-replicate-3: adding option 'node-uuid' for volume >> 'gfs-replicate-3' with value '85eb78cd-8ffa-49ca-b3e7-d5030bc3124d' >> [2016-02-01 15:03:05.065510] I [graph.c:269:gf_add_cmdline_options] >> 0-gfs-replicate-2: adding option 'node-uuid' for volume >> 'gfs-replicate-2' with value '85eb78cd-8ffa-49ca-b3e7-d5030bc3124d' >> [2016-02-01 15:03:05.065532] I [graph.c:269:gf_add_cmdline_options] >> 0-gfs-replicate-1: adding option 'node-uuid' for volume >> 'gfs-replicate-1' with value '85eb78cd-8ffa-49ca-b3e7-d5030bc3124d' >> [2016-02-01 15:03:05.065552] I [graph.c:269:gf_add_cmdline_options] >> 0-gfs-replicate-0: adding option 'node-uuid' for volume >> 'gfs-replicate-0' with value '85eb78cd-8ffa-49ca-b3e7-d5030bc3124d' >> [2016-02-01 15:03:05.065866] E [MSGID: 108040] [afr.c:418:init] >> 0-gfs-replicate-3: Unable to fetch afr pending changelogs. Is >> op-version >= 30707? [Invalid argument] >> [2016-02-01 15:03:05.066002] E [MSGID: 101019] >> [xlator.c:433:xlator_init] 0-gfs-replicate-3: Initialization of >> volume 'gfs-replicate-3' failed, review your volfile again >> [2016-02-01 15:03:05.066024] E [graph.c:322:glusterfs_graph_init] >> 0-gfs-replicate-3: initializing translator failed >> [2016-02-01 15:03:05.066045] E [graph.c:661:glusterfs_graph_activate] >> 0-graph: init failed >> [2016-02-01 15:03:05.066348] W [glusterfsd.c:1236:cleanup_and_exit] >> (-->0xbbbd9d64 <rpc_clnt_handle_reply+452> at >> /usr/pkg/lib/libgfrpc.so.0 -->0x8055f28 <mgmt_getspec_cbk+850> at >> /usr/pkg/sbin/glusterfs -->0x80518a4 <glusterfs_process_volfp+430> at >> /usr/pkg/sbin/glusterfs ) 0-: received signum (0), shutting down >> > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users