Network Attached Storage​

 View Only

 SamLogon: Account domain\login ID on client XX.XX.XX.XX failed to logon with error NT_STATUS_WRONG_PASSWORD (0xc000006a) - mapped to FSBStatus WrongPassword: this event, Id 1000034, (and similar ones) happened 187 times in the last 128.9 min on the CMB1.

Muneer Khan's profile image
Muneer Khan posted 01-25-2023 06:13
Hi Experts, 

we are facing very weird issue on Hitachi HNAS 5200, we are getting following event on HNAS and HNAS is becoming sluggish due to many hits of BAD Password.

HNAS-5200 CIFS: SamLogon: Account domain\login ID on client XX.XX.XX.XX failed to logon with error NT_STATUS_WRONG_PASSWORD (0xc000006a) - mapped to FSBStatus WrongPassword: this event, Id 1000034, (and similar ones) happened 187 times in the last 128.9 min on the CMB1.

not getting any events or autologin scripts, or any software which may be getting triggered for authentication  on DC, and on Domain also we are not getting any event for bad passwords.
in-fact there is group policy to locked the AD ID after certain wrong attempts. 

due to this wired behavior of HNAS 5200, the entire node is getting inaccessible.

any suggestion / work around to overcome on this issue is much appreciable.

Regards
MK
Albert Hagopian's profile image
Albert Hagopian

Hello Muneer: I see your account in SFDC and that you raised a case back in late Sept '22. In fact, the topic was brought to my attention by local Tech Expert in Oct.

From a very brief review, this topic has lineage to our client barring implementation.

Please do let me know if you cannot access this link; if not we can provide details to your account team SE.

Additionally, one can never have a "one size fits all" answer - ie, the event you posted may not correlate to the same root cause as another customer; according to case notes, beyond the client barring initiative - moving to Kerberos (from NTLM authentication) will also solve the follow-on errors issues.

Lastly, our support center provided methodologies to expand what we term the "paced event log" so that you can stochastically grep this log via SSC in order to see any event statuses that are not covered by the client barring. There was also an inquiry from your team as to the potential use of HNAS ELK integration, though I do not think that inquiry is quite relative to what is being asked in this context.

Muneer Khan's profile image
Muneer Khan
Hi, 

thanks for the revert, 
client barring is already implemented and surprising its not helping. 

the other observation is the IDs which are already deleted from Active Directory, however we still see event on HNAS . 

"2023-01-30 10:31:18.545+05:30 PacedEventLoggerImplementation(@0x00007F0E1A667000): paced: CIFS: Failed to resolve name:domain\username to SID with error NT_STATUS_NONE_MAPPED (0xc0000073) - mapped to FSBStatus NoUserMapping: this event, Id 1000029, (and similar ones) happened 350 times in the last 83.55 min on the CMB1." 

Also one of the observation from Hitachi onsite engineer is : session are not getting disconnected from Hitachi, its persistent. how do we tweak system to release and reconnect the session if user is ideal or change the  password and coming with new password.

why this event !
Albert Hagopian's profile image
Albert Hagopian

Muneer, I took a quick loot at diagnostics today and there are plenty of errors. What I cannot ascertain is whether this is relative to the request to move to Kerberos. It appears you may have moved on from client barring, so opening a new case would probably be wise.

you could investigate the CLI commands:

smb2-session-connection-list
connection --delete

<cifs-dc-errors for pnode 1>
Count Command Status FSBStatus
------ -------------------------- --------------------------------- ----------------
479050 LsarpcSid2Name NT_STATUS_NONE_MAPPED NoUserMapping
40624 LsarpcName2Sid NT_STATUS_NONE_MAPPED NoUserMapping

Muneer Khan's profile image
Muneer Khan
Thanks Albert, shared command helped.
the output is as follows 

IN-HOHNAS-1:$ cifs-dc-errors
Count Command Status FSBStatus
------ -------------------------- --------------------------------- ----------------
525098 LsarpcSid2Name NT_STATUS_NONE_MAPPED NoUserMapping
44317 LsarpcName2Sid NT_STATUS_NONE_MAPPED NoUserMapping
12762 SamrOpenUser NT_STATUS_UNSUCCESSFUL FailedToTransmit
6006 NetLogonSamLogonEx NT_STATUS_UNSUCCESSFUL FailedToTransmit
3305 NetLogonSamLogonEx NT_STATUS_NO_SUCH_USER NoSuchUser
1921 NetLogonSamLogonEx NT_STATUS_WRONG_PASSWORD WrongPassword
663 NetLogonSamLogonEx NT_STATUS_ACCOUNT_LOCKED_OUT AccountLockedOut
366 SessionSetupAndTreeConnect NT_STATUS_UNSUCCESSFUL FailedToTransmit
328 NetLogonSamLogonEx NT_STATUS_PASSWORD_EXPIRED PasswordExpired
78 LsarpcSid2Name NT_STATUS_UNSUCCESSFUL FailedToTransmit
49 LsarpcName2Sid NT_STATUS_UNSUCCESSFUL FailedToTransmit
16 LsarpcSid2Name NT_STATUS_NETWORK_SESSION_EXPIRED SessionExpired
4 NetLogonSamLogonEx NT_STATUS_ACCOUNT_DISABLED AccountDisabled
2 NetLogonSamLogonEx NT_STATUS_ACCOUNT_EXPIRED AccountExpired

is there a way in HNAS to identify from which systems some process trying to use deleted IDs to access HNAS , I dont see any event on Active  Directory which for deleted IDs accessing/trying to authenticate on AD.  

regards
MK
Albert Hagopian's profile image
Albert Hagopian

Muneer, 

Please open a new case with GSC for continued talks.

This forum is not in place to circumvent proper support; what info you seek is esoteric and no one on this forum can assist in diagnosis of your system inquiries (nor should they). I've asked our India Tech Expert to help guide expectations.

Andrew J. Romero's profile image
Andrew J. Romero

Hi AL,  Hi Muneer

I'm a little late to the forum,  I have been using HNAS since the
BlueArc days; but, I just discovered this interesting forum

Muneer I currently have non-zero cifs-dc-errors error counts (see below).
However, I currently have no sluggishness / performance issues.
These error count numbers may just be normal noise in a large environment.

Note: 187 bad password events in 2 hours shouldn't cause sluggishness

What is ( or was ) the nature of your performance issue.
  - basic sluggishness ( example: you run ls / dir from a CIFS/NFS client and it takes
    several seconds before you get a response )
  - slower than normal throughput ( you normally see 200MB/s writes; but, now you
    see 10 MB/s )

Over the years I have had occasional performance issues, 99.9 % fell into
one of these two (non-mysterious ) general categories

 - NAS getting overloaded by users ( I have a lot of users )
   The most basic way to detect an overload is to look at this on the SMU:

   Status & Monitoring > Performance Graphs > Running Network Receive Fibers

   If the numbers are consistently above ~50, you are getting slightly oveloaded,
   If the numbers are "flat-topped / clipped at 400 , you are getting max overloaded.
   After determining : YES, my HNAS is overloaded, pin-pointing the culprit is
   somewhat of an art. ( I find the CLI connection and file access count tools to
   be a great help here )

 - Bad optic (LAN or SAN ). Check errors from HNAS and LAN / SAN switch perspective


If you have already solved your problem, let the forum know the solution.

Andy R.

Andrew J. Romero's profile image
Andrew J. Romero

Here are my non-zero cifs-dc-errors
( don't seem to be an issue for me ... just noise )

file-svc-a-1:$ pn 1 cifs-dc-errors
  Count  Command   Status     FSBStatus
  -------  --------------------------  
  1157352  LsarpcName2Sid  NT_STATUS_NONE_MAPPED  NoUserMapping
  744443 LsarpcSid2Name  NT_STATUS_NONE_MAPPED  NoUserMapping
  131785 NetLogonSamLogonEx  NT_STATUS_WRONG_PASSWORD   WrongPassword
  50320  NetLogonSamLogonEx  NT_STATUS_NO_SUCH_USER   NoSuchUser
  3174 LsarpcSid2Name  NT_STATUS_UNSUCCESSFUL   FailedToTransmit
  954  NetLogonSamLogonEx  NT_STATUS_ACCOUNT_LOCKED_OUT AccountLockedOut
  900  LsarpcName2Sid  NT_STATUS_UNSUCCESSFUL   FailedToTransmit
  91 NetLogonSamLogonEx  NT_STATUS_UNSUCCESSFUL   FailedToTransmit
  50 LsarpcSid2Name  NT_STATUS_NETWORK_SESSION_EXPIRED  SessionExpired
  10 LsarpcName2Sid  NT_STATUS_NETWORK_SESSION_EXPIRED  SessionExpired
  9  NetLogonSamLogonEx  NT_STATUS_NETWORK_SESSION_EXPIRED  SessionExpired
  5  SessionSetupAndTreeConnect  NT_STATUS_UNSUCCESSFUL   FailedToTransmit
  3  NegProt   NT_STATUS_UNSUCCESSFUL   ServerError
  2  LsarpcSid2Name  NT_STATUS_INVALID_SERVER_STATE InvalidServerState
  2  LsarpcName2Sid  NT_STATUS_INVALID_SERVER_STATE InvalidServerState
  2  LsarpcSid2Name  NT_STATUS_ACCESS_DENIED  OperationHasTimedOut
  2  NetLogonSamLogonEx  NT_STATUS_ACCESS_DENIED  OperationHasTimedOut

Muneer Khan's profile image
Muneer Khan

Thanks All for your positive revert, unfortunately the issue is not yet resolved. 

the pattern is once users change their password (Active Directory - ID) after few mins the machine(endpoint), from which users changed the password start sending the bad password requests and HNAS 5200 become inaccessible. (the network receive fibers shoots to max) and post endpoint reboot HNAS get normalize. 

we are clueless why its happening, as we don't see any log in AD for wrong password hit and user account/ID is not getting locked (as we have password policy).

Regards

MK