Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm currently using DRBD to replicate a samba file server accross two machines. Sometimes writes fail on the master, when this happens the resource status (using cat /proc/drbd) is:

0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate A r---n

When things work fine the status is:

0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate A r----

The only difference I can see is the 'n' at the end of the line. I don't think this is related to congestion since it occurs when there's no activity on the resource. Disconnecting and reconnecting using drbdadm solves the problem.

So, my question is: what does that 'n' means?

Thank you for your help!

share|improve this question
add comment

1 Answer

up vote 4 down vote accepted

I haven't found some documentation about those bits, so I've dug in the drbd-8.3.4 source code. The 'n' is in fact related to sync congestion. Here's the code:

this is the call to update the /proc/drbd

in drdb/drbd_main.c

seq_printf(seq,
201                            "%2d: cs:%s ro:%s/%s ds:%s/%s %c %c%c%c%c%c\n"
202                            "    ns:%u nr:%u dw:%u dr:%u al:%u bm:%u "
203                            "lo:%d pe:%d ua:%d ap:%d ep:%d wo:%c",
204                            i, sn,
205                            drbd_role_str(mdev->state.role),
206                            drbd_role_str(mdev->state.peer),
207                            drbd_disk_str(mdev->state.disk),
208                            drbd_disk_str(mdev->state.pdsk),
209                            (mdev->net_conf == NULL ? ' ' :
210                             (mdev->net_conf->wire_protocol - DRBD_PROT_A+'A')),
211                            mdev->state.susp ? 's' : 'r',
212                            mdev->state.aftr_isp ? 'a' : '-',
213                            mdev->state.peer_isp ? 'p' : '-',
214                            mdev->state.user_isp ? 'u' : '-',
215                            mdev->congestion_reason ?: '-',
216                            mdev->send_cnt/2,
217                            mdev->recv_cnt/2,
218                            mdev->writ_cnt/2,
219                            mdev->read_cnt/2,
220                            mdev->al_writ_cnt,
221                            mdev->bm_writ_cnt,
222                            atomic_read(&mdev->local_cnt),
223                            atomic_read(&mdev->ap_pending_cnt) +
224                            atomic_read(&mdev->rs_pending_cnt),
225                            atomic_read(&mdev->unacked_cnt),
226                            atomic_read(&mdev->ap_bio_cnt),
227                            mdev->epochs,
228                            write_ordering_chars[mdev->write_ordering]
229                         );

Which means that the 'n' field comes from the congestion_reason. And this is set in the following code (line 3171):

in drdb/drbd_main.c

3140 /**                       
3141  * drbd_congested() - Callback for pdflush
3142  * @congested_data:     User data
3143  * @bdi_bits:           Bits pdflush is currently interested in
3144  *                        
3145  * Returns 1<<BDI_async_congested and/or 1<<BDI_sync_congested if we are congested.
3146  */                       
3147 static int drbd_congested(void *congested_data, int bdi_bits)
3148 {                         
3149         struct drbd_conf *mdev = congested_data;
3150         struct request_queue *q;
3151         char reason = '-';
3152         int r = 0;        
3153                           
3154         if (!__inc_ap_bio_cond(mdev)) {
3155                 /* DRBD has frozen IO */
3156                 r = bdi_bits;
3157                 reason = 'd';     
3158                 goto out;
3159         }      
3160                    
3161         if (get_ldev(mdev)) {
3162                 q = bdev_get_queue(mdev->ldev->backing_bdev);
3163                 r = bdi_congested(&q->backing_dev_info, bdi_bits);
3164                 put_ldev(mdev);
3165                 if (r)            
3166                         reason = 'b';   
3167         }                         
3168                                   
3169         if (bdi_bits & (1 << BDI_async_congested) && test_bit(NET_CONGESTED, &mdev->flags)) {
3170                 r |= (1 << BDI_async_congested);
3171                 reason = reason == 'b' ? 'a' : 'n';
3172         }              
3173                        
3174 out:           
3175         mdev->congestion_reason = reason;
3176         return r;
3177 }

Hope this helps

share|improve this answer
    
Thanks, I've noticed that when this happens the Send-Q of the TCP connections doesn't flush. However the link between the two nodes doesn't appear to be congested at all. –  Guillaume Jul 31 '10 at 5:41
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.