Login
Log in using an SSO provider:
Fedora Account System
Red Hat Associate
Red Hat Customer
Login using a Red Hat Bugzilla account
Forgot Password
Create an Account
Red Hat Bugzilla – Attachment 1976120 Details for
Bug 2221772
Automatic failover - Pacemaker 2.0 , on-fail=demote with sql server 3node replicas, stat-failure-is-fatal not being adhered to
Home
New
Search
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh89 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
[?]
This site requires JavaScript to be enabled to function correctly, please enable it.
Pacemaker logs
pacemaker3.log (text/plain), 292.17 KB, created by
Aravind Mahadevan
on 2023-07-17 05:30:22 UTC
(
hide
)
Description:
Pacemaker logs
Filename:
MIME Type:
Creator:
Aravind Mahadevan
Created:
2023-07-17 05:30:22 UTC
Size:
292.17 KB
patch
obsolete
>Jul 14 17:07:12 ag(ag_cluster)[890196]: INFO: monitor: 2023/07/14 17:07:12 Monitor Caller is: monitor. >Jul 14 17:07:12 ag(ag_cluster)[890196]: INFO: monitor: 2023/07/14 17:07:12 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:07:12 ag(ag_cluster)[890196]: INFO: monitor: 2023/07/14 17:07:12 Replica is RESOLVING (0) >Jul 14 17:07:12 ag(ag_cluster)[890196]: INFO: monitor: 2023/07/14 17:07:12 Instance name is server3. >Jul 14 17:07:12 ag(ag_cluster)[890196]: INFO: monitor: 2023/07/14 17:07:12 Current master is . >Jul 14 17:07:12 ag(ag_cluster)[890196]: INFO: monitor: PROMOTION_SCORE: 10 >Jul 14 17:07:12 ag(ag_cluster)[890196]: INFO: lease_expiry after monitor update: >Jul 14 17:07:12 ag(ag_cluster)[890196]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:07:12 ag(ag_cluster)[890196]: INFO: ag_cluster monitor : 0 >Jul 14 17:07:12 ag(ag_cluster)[890350]: INFO: mssql_validate >Jul 14 17:07:12 ag(ag_cluster)[890350]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:07:12 ag(ag_cluster)[890350]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:07:12 ag(ag_cluster)[890350]: INFO: Resource agent invoked with: notify >Jul 14 17:07:12 ag(ag_cluster)[890350]: INFO: mssql_notify post-start >Jul 14 17:07:12 ag(ag_cluster)[890350]: INFO: ag_cluster notify : 0 >Jul 14 17:07:12 ag(ag_cluster)[890396]: INFO: mssql_validate >Jul 14 17:07:12 ag(ag_cluster)[890396]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:07:12 ag(ag_cluster)[890396]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:07:12 ag(ag_cluster)[890396]: INFO: Resource agent invoked with: notify >Jul 14 17:07:12 ag(ag_cluster)[890396]: INFO: mssql_notify pre-promote >Jul 14 17:07:12 ag(ag_cluster)[890396]: INFO: notify: 2023/07/14 17:07:12 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-promote]; connection-timeout [30]; health-threshold [3]; action [pre-promote] >Jul 14 17:07:12 ag(ag_cluster)[890396]: INFO: notify: 2023/07/14 17:07:12 From RetryExecute - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:07:12 ag(ag_cluster)[890396]: INFO: notify: 2023/07/14 17:07:12 Connected to the instance at localhost:1433 >Jul 14 17:07:17 ag(ag_cluster)[890396]: INFO: notify: 2023/07/14 17:07:17 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:07:17 ag(ag_cluster)[890396]: INFO: notify: 2023/07/14 17:07:17 Sequence number is 17F:F (1644972474383) >Jul 14 17:07:17 ag(ag_cluster)[890396]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:07:23 ag(ag_cluster)[890396]: INFO: ag_cluster notify : 0 >Jul 14 17:07:23 ag(ag_cluster)[890505]: INFO: mssql_validate >Jul 14 17:07:23 ag(ag_cluster)[890505]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:07:23 ag(ag_cluster)[890505]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:07:23 ag(ag_cluster)[890505]: INFO: Resource agent invoked with: monitor >Jul 14 17:07:23 ag(ag_cluster)[890505]: INFO: mssql_monitor >Jul 14 17:07:23 ag(ag_cluster)[890505]: INFO: monitor: 2023/07/14 17:07:23 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:07:23 ag(ag_cluster)[890505]: INFO: monitor: 2023/07/14 17:07:23 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master []; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:07:23 ag(ag_cluster)[890505]: INFO: monitor: 2023/07/14 17:07:23 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:07:23 ag(ag_cluster)[890505]: INFO: monitor: 2023/07/14 17:07:23 Connected to the instance at localhost:1433 >Jul 14 17:07:28 ag(ag_cluster)[890505]: INFO: monitor: 2023/07/14 17:07:28 Monitor Caller is: monitor. >Jul 14 17:07:28 ag(ag_cluster)[890505]: INFO: monitor: 2023/07/14 17:07:28 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:07:28 ag(ag_cluster)[890505]: INFO: monitor: 2023/07/14 17:07:28 Replica is RESOLVING (0) >Jul 14 17:07:28 ag(ag_cluster)[890505]: INFO: monitor: 2023/07/14 17:07:28 Instance name is server3. >Jul 14 17:07:28 ag(ag_cluster)[890505]: INFO: monitor: 2023/07/14 17:07:28 Current master is . >Jul 14 17:07:28 ag(ag_cluster)[890505]: INFO: monitor: PROMOTION_SCORE: 10 >Jul 14 17:07:28 ag(ag_cluster)[890505]: INFO: lease_expiry after monitor update: >Jul 14 17:07:28 ag(ag_cluster)[890505]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:07:28 ag(ag_cluster)[890505]: INFO: ag_cluster monitor : 0 >Jul 14 17:07:28 ag(ag_cluster)[890659]: INFO: mssql_validate >Jul 14 17:07:28 ag(ag_cluster)[890659]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:07:28 ag(ag_cluster)[890659]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:07:28 ag(ag_cluster)[890659]: INFO: Resource agent invoked with: monitor >Jul 14 17:07:28 ag(ag_cluster)[890659]: INFO: mssql_monitor >Jul 14 17:07:28 ag(ag_cluster)[890659]: INFO: monitor: 2023/07/14 17:07:28 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:07:28 ag(ag_cluster)[890659]: INFO: monitor: 2023/07/14 17:07:28 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master []; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:07:28 ag(ag_cluster)[890659]: INFO: monitor: 2023/07/14 17:07:28 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:07:28 ag(ag_cluster)[890659]: INFO: monitor: 2023/07/14 17:07:28 Connected to the instance at localhost:1433 >Jul 14 17:07:33 ag(ag_cluster)[890659]: INFO: monitor: 2023/07/14 17:07:33 Monitor Caller is: monitor. >Jul 14 17:07:33 ag(ag_cluster)[890659]: INFO: monitor: 2023/07/14 17:07:33 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:07:33 ag(ag_cluster)[890659]: INFO: monitor: 2023/07/14 17:07:33 Replica is RESOLVING (0) >Jul 14 17:07:33 ag(ag_cluster)[890659]: INFO: monitor: 2023/07/14 17:07:33 Instance name is server3. >Jul 14 17:07:33 ag(ag_cluster)[890659]: INFO: monitor: 2023/07/14 17:07:33 Current master is . >Jul 14 17:07:33 ag(ag_cluster)[890659]: INFO: monitor: PROMOTION_SCORE: 10 >Jul 14 17:07:33 ag(ag_cluster)[890659]: INFO: lease_expiry after monitor update: >Jul 14 17:07:33 ag(ag_cluster)[890659]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:07:33 ag(ag_cluster)[890659]: INFO: ag_cluster monitor : 0 >Jul 14 17:07:33 ag(ag_cluster)[890822]: INFO: mssql_validate >Jul 14 17:07:33 ag(ag_cluster)[890822]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:07:33 ag(ag_cluster)[890822]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:07:33 ag(ag_cluster)[890822]: INFO: Resource agent invoked with: notify >Jul 14 17:07:33 ag(ag_cluster)[890822]: INFO: mssql_notify pre-promote >Jul 14 17:07:33 ag(ag_cluster)[890822]: INFO: notify: 2023/07/14 17:07:33 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-promote]; connection-timeout [30]; health-threshold [3]; action [pre-promote] >Jul 14 17:07:33 ag(ag_cluster)[890822]: INFO: notify: 2023/07/14 17:07:33 From RetryExecute - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:07:33 ag(ag_cluster)[890822]: INFO: notify: 2023/07/14 17:07:33 Connected to the instance at localhost:1433 >Jul 14 17:07:38 ag(ag_cluster)[890822]: INFO: notify: 2023/07/14 17:07:38 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:07:38 ag(ag_cluster)[890822]: INFO: notify: 2023/07/14 17:07:38 Sequence number is 17F:F (1644972474383) >Jul 14 17:07:38 ag(ag_cluster)[890822]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:07:44 ag(ag_cluster)[890822]: INFO: ag_cluster notify : 0 >Jul 14 17:07:44 ag(ag_cluster)[890932]: INFO: mssql_validate >Jul 14 17:07:44 ag(ag_cluster)[890932]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:07:44 ag(ag_cluster)[890932]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:07:44 ag(ag_cluster)[890932]: INFO: Resource agent invoked with: monitor >Jul 14 17:07:44 ag(ag_cluster)[890932]: INFO: mssql_monitor >Jul 14 17:07:44 ag(ag_cluster)[890932]: INFO: monitor: 2023/07/14 17:07:44 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:07:44 ag(ag_cluster)[890932]: INFO: monitor: 2023/07/14 17:07:44 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server1]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:07:44 ag(ag_cluster)[890932]: INFO: monitor: 2023/07/14 17:07:44 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:07:44 ag(ag_cluster)[890932]: INFO: monitor: 2023/07/14 17:07:44 Connected to the instance at localhost:1433 >Jul 14 17:07:49 ag(ag_cluster)[890932]: INFO: monitor: 2023/07/14 17:07:49 Monitor Caller is: monitor. >Jul 14 17:07:49 ag(ag_cluster)[890932]: INFO: monitor: 2023/07/14 17:07:49 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:07:49 ag(ag_cluster)[890932]: INFO: monitor: 2023/07/14 17:07:49 Replica is RESOLVING (0) >Jul 14 17:07:49 ag(ag_cluster)[890932]: INFO: monitor: 2023/07/14 17:07:49 Instance name is server3. >Jul 14 17:07:49 ag(ag_cluster)[890932]: INFO: monitor: 2023/07/14 17:07:49 Current master is server1. >Jul 14 17:07:49 ag(ag_cluster)[890932]: INFO: monitor: 2023/07/14 17:07:49 Setting the role to Secondary. >Jul 14 17:07:49 ag(ag_cluster)[890932]: INFO: monitor: 2023/07/14 17:07:49 Setting replica to SECONDARY role... >Jul 14 17:07:49 ag(ag_cluster)[890932]: INFO: monitor: 2023/07/14 17:07:49 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:07:49 ag(ag_cluster)[890932]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:07:49 ag(ag_cluster)[890932]: INFO: lease_expiry after monitor update: >Jul 14 17:07:49 ag(ag_cluster)[890932]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:07:49 ag(ag_cluster)[890932]: INFO: ag_cluster monitor : 0 >Jul 14 17:07:49 ag(ag_cluster)[891104]: INFO: mssql_validate >Jul 14 17:07:49 ag(ag_cluster)[891104]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:07:49 ag(ag_cluster)[891104]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:07:49 ag(ag_cluster)[891104]: INFO: Resource agent invoked with: monitor >Jul 14 17:07:49 ag(ag_cluster)[891104]: INFO: mssql_monitor >Jul 14 17:07:49 ag(ag_cluster)[891104]: INFO: monitor: 2023/07/14 17:07:49 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:07:49 ag(ag_cluster)[891104]: INFO: monitor: 2023/07/14 17:07:49 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server1]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:07:49 ag(ag_cluster)[891104]: INFO: monitor: 2023/07/14 17:07:49 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:07:49 ag(ag_cluster)[891104]: INFO: monitor: 2023/07/14 17:07:49 Connected to the instance at localhost:1433 >Jul 14 17:07:54 ag(ag_cluster)[891104]: INFO: monitor: 2023/07/14 17:07:54 Monitor Caller is: monitor. >Jul 14 17:07:54 ag(ag_cluster)[891104]: INFO: monitor: 2023/07/14 17:07:54 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:07:54 ag(ag_cluster)[891104]: INFO: monitor: 2023/07/14 17:07:54 Replica is RESOLVING (0) >Jul 14 17:07:54 ag(ag_cluster)[891104]: INFO: monitor: 2023/07/14 17:07:54 Instance name is server3. >Jul 14 17:07:54 ag(ag_cluster)[891104]: INFO: monitor: 2023/07/14 17:07:54 Current master is server1. >Jul 14 17:07:54 ag(ag_cluster)[891104]: INFO: monitor: 2023/07/14 17:07:54 Setting the role to Secondary. >Jul 14 17:07:54 ag(ag_cluster)[891104]: INFO: monitor: 2023/07/14 17:07:54 Setting replica to SECONDARY role... >Jul 14 17:07:54 ag(ag_cluster)[891104]: INFO: monitor: 2023/07/14 17:07:54 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:07:54 ag(ag_cluster)[891104]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:07:54 ag(ag_cluster)[891104]: INFO: lease_expiry after monitor update: >Jul 14 17:07:54 ag(ag_cluster)[891104]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:07:54 ag(ag_cluster)[891104]: INFO: ag_cluster monitor : 0 >Jul 14 17:07:54 ag(ag_cluster)[891276]: INFO: mssql_validate >Jul 14 17:07:54 ag(ag_cluster)[891276]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:07:54 ag(ag_cluster)[891276]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:07:54 ag(ag_cluster)[891276]: INFO: Resource agent invoked with: notify >Jul 14 17:07:54 ag(ag_cluster)[891276]: INFO: mssql_notify post-promote >Jul 14 17:07:54 ag(ag_cluster)[891276]: INFO: notify: 2023/07/14 17:07:54 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-promote]; connection-timeout [30]; health-threshold [3]; action [post-promote] >Jul 14 17:07:54 ag(ag_cluster)[891276]: INFO: notify: 2023/07/14 17:07:54 [DEBUG] AG Helper PostPromote Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:07:54 ag(ag_cluster)[891276]: INFO: notify: 2023/07/14 17:07:54 Replica is RESOLVING (0) >Jul 14 17:07:54 ag(ag_cluster)[891276]: INFO: ag_cluster notify : 0 >Jul 14 17:07:55 ag(ag_cluster)[891352]: INFO: mssql_validate >Jul 14 17:07:55 ag(ag_cluster)[891352]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:07:55 ag(ag_cluster)[891352]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:07:55 ag(ag_cluster)[891352]: INFO: Resource agent invoked with: notify >Jul 14 17:07:55 ag(ag_cluster)[891352]: INFO: mssql_notify pre-demote >Jul 14 17:07:55 ag(ag_cluster)[891352]: INFO: ag_cluster notify : 0 >Jul 14 17:07:55 ag(ag_cluster)[891397]: INFO: mssql_validate >Jul 14 17:07:55 ag(ag_cluster)[891397]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:07:55 ag(ag_cluster)[891397]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:07:55 ag(ag_cluster)[891397]: INFO: Resource agent invoked with: notify >Jul 14 17:07:55 ag(ag_cluster)[891397]: INFO: mssql_notify post-demote >Jul 14 17:07:55 ag(ag_cluster)[891397]: INFO: ag_cluster notify : 0 >Jul 14 17:07:55 ag(ag_cluster)[891442]: INFO: mssql_validate >Jul 14 17:07:55 ag(ag_cluster)[891442]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:07:55 ag(ag_cluster)[891442]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:07:55 ag(ag_cluster)[891442]: INFO: Resource agent invoked with: notify >Jul 14 17:07:55 ag(ag_cluster)[891442]: INFO: mssql_notify pre-stop >Jul 14 17:07:55 ag(ag_cluster)[891442]: INFO: ag_cluster notify : 0 >Jul 14 17:07:55 ag(ag_cluster)[891487]: INFO: mssql_validate >Jul 14 17:07:55 ag(ag_cluster)[891487]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:07:55 ag(ag_cluster)[891487]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:07:55 ag(ag_cluster)[891487]: INFO: Resource agent invoked with: notify >Jul 14 17:07:55 ag(ag_cluster)[891487]: INFO: mssql_notify post-stop >Jul 14 17:07:55 ag(ag_cluster)[891487]: INFO: notify: 2023/07/14 17:07:55 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-stop]; connection-timeout [30]; health-threshold [3]; action [post-stop] >Jul 14 17:07:55 ag(ag_cluster)[891487]: INFO: notify: 2023/07/14 17:07:55 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:07:55 ag(ag_cluster)[891487]: INFO: notify: 2023/07/14 17:07:55 Replica is RESOLVING (0) >Jul 14 17:07:55 ag(ag_cluster)[891487]: INFO: ag_cluster notify : 0 >Jul 14 17:07:55 ag(ag_cluster)[891562]: INFO: mssql_validate >Jul 14 17:07:55 ag(ag_cluster)[891562]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:07:55 ag(ag_cluster)[891562]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:07:55 ag(ag_cluster)[891562]: INFO: Resource agent invoked with: notify >Jul 14 17:07:55 ag(ag_cluster)[891562]: INFO: mssql_notify pre-start >Jul 14 17:07:55 ag(ag_cluster)[891562]: INFO: notify: 2023/07/14 17:07:55 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:07:55 ag(ag_cluster)[891562]: INFO: notify: 2023/07/14 17:07:55 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:07:55 ag(ag_cluster)[891562]: INFO: notify: 2023/07/14 17:07:55 Replica is RESOLVING (0) >Jul 14 17:07:55 ag(ag_cluster)[891562]: INFO: notify: 2023/07/14 17:07:55 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:07:55 ag(ag_cluster)[891562]: INFO: notify: 2023/07/14 17:07:55 Sequence number is 17F:F (1644972474383) >Jul 14 17:07:55 ag(ag_cluster)[891562]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:08:00 ag(ag_cluster)[891562]: INFO: ag_cluster notify : 0 >Jul 14 17:08:00 ag(ag_cluster)[891663]: INFO: mssql_validate >Jul 14 17:08:00 ag(ag_cluster)[891663]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:08:00 ag(ag_cluster)[891663]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:08:00 ag(ag_cluster)[891663]: INFO: Resource agent invoked with: notify >Jul 14 17:08:00 ag(ag_cluster)[891663]: INFO: mssql_notify pre-stop >Jul 14 17:08:00 ag(ag_cluster)[891663]: INFO: ag_cluster notify : 0 >Jul 14 17:08:00 ag(ag_cluster)[891708]: INFO: mssql_validate >Jul 14 17:08:00 ag(ag_cluster)[891708]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:08:00 ag(ag_cluster)[891708]: INFO: OCF_RESKEY_CRM_meta_timeout value: 10000 >Jul 14 17:08:00 ag(ag_cluster)[891708]: INFO: Resource agent invoked with: stop >Jul 14 17:08:00 ag(ag_cluster)[891708]: INFO: mssql_stop >Jul 14 17:08:01 ag(ag_cluster)[891708]: INFO: stop: 2023/07/14 17:08:01 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-stop]; connection-timeout [30]; health-threshold [3]; action [stop] >Jul 14 17:08:01 ag(ag_cluster)[891708]: INFO: stop: 2023/07/14 17:08:01 [DEBUG] AG Helper Stop Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:08:01 ag(ag_cluster)[891708]: INFO: stop: 2023/07/14 17:08:01 Replica is RESOLVING (0) >Jul 14 17:08:01 ag(ag_cluster)[891708]: INFO: ag_cluster stop : 0 >Jul 14 17:08:06 ag(ag_cluster)[891789]: INFO: mssql_validate >Jul 14 17:08:06 ag(ag_cluster)[891789]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:08:06 ag(ag_cluster)[891789]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:08:06 ag(ag_cluster)[891789]: INFO: Resource agent invoked with: start >Jul 14 17:08:06 ag(ag_cluster)[891789]: INFO: mssql_start >Jul 14 17:08:06 ag(ag_cluster)[891789]: INFO: start: 2023/07/14 17:08:06 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-start]; connection-timeout [30]; health-threshold [3]; action [start] >Jul 14 17:08:06 ag(ag_cluster)[891789]: INFO: start: 2023/07/14 17:08:06 ag-helper invoked with sequence-numbers [...]; required-synchronized-secondaries-to-commit [-1]; current-master []; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [62] >Jul 14 17:08:06 ag(ag_cluster)[891789]: INFO: start: 2023/07/14 17:08:06 From RetryExecute - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:08:06 ag(ag_cluster)[891789]: INFO: start: 2023/07/14 17:08:06 Connected to the instance at localhost:1433 >Jul 14 17:08:11 ag(ag_cluster)[891789]: INFO: start: 2023/07/14 17:08:11 [DEBUG] AG Helper Start Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:08:11 ag(ag_cluster)[891789]: INFO: start: 2023/07/14 17:08:11 Replica is RESOLVING (0) >Jul 14 17:08:11 ag(ag_cluster)[891789]: INFO: start: 2023/07/14 17:08:11 Sequence number is 17F:F (1644972474383) >Jul 14 17:08:11 ag(ag_cluster)[891789]: INFO: start: 2023/07/14 17:08:11 Verifying replica's sequence number vs all sequence numbers... >Jul 14 17:08:11 ag(ag_cluster)[891789]: INFO: start: 2023/07/14 17:08:11 Sequence number line [name="ag_cluster-sequence-number" host="server1" value=""] >Jul 14 17:08:11 ag(ag_cluster)[891789]: INFO: start: 2023/07/14 17:08:11 Line does not match expected syntax. Ignoring. >Jul 14 17:08:11 ag(ag_cluster)[891789]: INFO: start: 2023/07/14 17:08:11 Sequence number line [name="ag_cluster-sequence-number" host="server3" value="1644972474383"] >Jul 14 17:08:11 ag(ag_cluster)[891789]: INFO: start: 2023/07/14 17:08:11 Sequence number line [name="ag_cluster-sequence-number" host="server2" value="52892522250255"] >Jul 14 17:08:11 ag(ag_cluster)[891789]: INFO: start: 2023/07/14 17:08:11 2 sequence numbers were found >Jul 14 17:08:11 ag(ag_cluster)[891789]: INFO: start: 2023/07/14 17:08:11 Max sequence number is 301B:F (52892522250255) >Jul 14 17:08:11 ag(ag_cluster)[891789]: INFO: start: 2023/07/14 17:08:11 Replica has sequence number 17F:F (1644972474383) but max sequence number is 301B:F (52892522250255), so it cannot be promoted >Jul 14 17:08:11 ag(ag_cluster)[891789]: INFO: start: PROMOTION_SCORE: -INFINITY >Jul 14 17:08:11 ag(ag_cluster)[891789]: INFO: start: 2023/07/14 17:08:11 Monitor Caller is: start. >Jul 14 17:08:11 ag(ag_cluster)[891789]: INFO: start: 2023/07/14 17:08:11 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:08:11 ag(ag_cluster)[891789]: INFO: start: 2023/07/14 17:08:11 Replica is RESOLVING (0) >Jul 14 17:08:11 ag(ag_cluster)[891789]: INFO: start: 2023/07/14 17:08:11 Instance name is server3. >Jul 14 17:08:11 ag(ag_cluster)[891789]: INFO: start: 2023/07/14 17:08:11 Current master is . >Jul 14 17:08:11 ag(ag_cluster)[891789]: INFO: ag_cluster start : 0 >Jul 14 17:08:11 ag(ag_cluster)[891988]: INFO: mssql_validate >Jul 14 17:08:11 ag(ag_cluster)[891988]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:08:11 ag(ag_cluster)[891988]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:08:11 ag(ag_cluster)[891988]: INFO: Resource agent invoked with: notify >Jul 14 17:08:11 ag(ag_cluster)[891988]: INFO: mssql_notify post-start >Jul 14 17:08:11 ag(ag_cluster)[891988]: INFO: ag_cluster notify : 0 >Jul 14 17:08:12 ag(ag_cluster)[892034]: INFO: mssql_validate >Jul 14 17:08:12 ag(ag_cluster)[892034]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:08:12 ag(ag_cluster)[892034]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:08:12 ag(ag_cluster)[892034]: INFO: Resource agent invoked with: notify >Jul 14 17:08:12 ag(ag_cluster)[892034]: INFO: mssql_notify pre-stop >Jul 14 17:08:12 ag(ag_cluster)[892034]: INFO: ag_cluster notify : 0 >Jul 14 17:08:12 ag(ag_cluster)[892079]: INFO: mssql_validate >Jul 14 17:08:12 ag(ag_cluster)[892079]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:08:12 ag(ag_cluster)[892079]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:08:12 ag(ag_cluster)[892079]: INFO: Resource agent invoked with: notify >Jul 14 17:08:12 ag(ag_cluster)[892079]: INFO: mssql_notify post-stop >Jul 14 17:08:12 ag(ag_cluster)[892079]: INFO: notify: 2023/07/14 17:08:12 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-stop]; connection-timeout [30]; health-threshold [3]; action [post-stop] >Jul 14 17:08:12 ag(ag_cluster)[892079]: INFO: notify: 2023/07/14 17:08:12 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:08:12 ag(ag_cluster)[892079]: INFO: notify: 2023/07/14 17:08:12 Replica is RESOLVING (0) >Jul 14 17:08:12 ag(ag_cluster)[892079]: INFO: ag_cluster notify : 0 >Jul 14 17:08:12 ag(ag_cluster)[892153]: INFO: mssql_validate >Jul 14 17:08:12 ag(ag_cluster)[892153]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:08:12 ag(ag_cluster)[892153]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:08:12 ag(ag_cluster)[892153]: INFO: Resource agent invoked with: notify >Jul 14 17:08:12 ag(ag_cluster)[892153]: INFO: mssql_notify pre-promote >Jul 14 17:08:12 ag(ag_cluster)[892153]: INFO: notify: 2023/07/14 17:08:12 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-promote]; connection-timeout [30]; health-threshold [3]; action [pre-promote] >Jul 14 17:08:12 ag(ag_cluster)[892153]: INFO: notify: 2023/07/14 17:08:12 From RetryExecute - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:08:12 ag(ag_cluster)[892153]: INFO: notify: 2023/07/14 17:08:12 Connected to the instance at localhost:1433 >Jul 14 17:08:17 ag(ag_cluster)[892153]: INFO: notify: 2023/07/14 17:08:17 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:08:17 ag(ag_cluster)[892153]: INFO: notify: 2023/07/14 17:08:17 Sequence number is 17F:F (1644972474383) >Jul 14 17:08:17 ag(ag_cluster)[892153]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:08:22 ag(ag_cluster)[892153]: INFO: ag_cluster notify : 0 >Jul 14 17:08:22 ag(ag_cluster)[892262]: INFO: mssql_validate >Jul 14 17:08:22 ag(ag_cluster)[892262]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:08:22 ag(ag_cluster)[892262]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:08:22 ag(ag_cluster)[892262]: INFO: Resource agent invoked with: notify >Jul 14 17:08:22 ag(ag_cluster)[892262]: INFO: mssql_notify pre-promote >Jul 14 17:08:22 ag(ag_cluster)[892262]: INFO: notify: 2023/07/14 17:08:22 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-promote]; connection-timeout [30]; health-threshold [3]; action [pre-promote] >Jul 14 17:08:22 ag(ag_cluster)[892262]: INFO: notify: 2023/07/14 17:08:22 From RetryExecute - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:08:22 ag(ag_cluster)[892262]: INFO: notify: 2023/07/14 17:08:22 Connected to the instance at localhost:1433 >Jul 14 17:08:27 ag(ag_cluster)[892262]: INFO: notify: 2023/07/14 17:08:27 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:08:27 ag(ag_cluster)[892262]: INFO: notify: 2023/07/14 17:08:27 Sequence number is 17F:F (1644972474383) >Jul 14 17:08:27 ag(ag_cluster)[892262]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:08:32 ag(ag_cluster)[892262]: INFO: ag_cluster notify : 0 >Jul 14 17:08:38 ag(ag_cluster)[892376]: INFO: mssql_validate >Jul 14 17:08:38 ag(ag_cluster)[892376]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:08:38 ag(ag_cluster)[892376]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:08:38 ag(ag_cluster)[892376]: INFO: Resource agent invoked with: notify >Jul 14 17:08:38 ag(ag_cluster)[892376]: INFO: mssql_notify post-promote >Jul 14 17:08:38 ag(ag_cluster)[892376]: INFO: notify: 2023/07/14 17:08:38 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-promote]; connection-timeout [30]; health-threshold [3]; action [post-promote] >Jul 14 17:08:38 ag(ag_cluster)[892376]: INFO: notify: 2023/07/14 17:08:38 [DEBUG] AG Helper PostPromote Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:08:38 ag(ag_cluster)[892376]: INFO: notify: 2023/07/14 17:08:38 Replica is RESOLVING (0) >Jul 14 17:08:38 ag(ag_cluster)[892376]: INFO: ag_cluster notify : 0 >Jul 14 17:08:38 ag(ag_cluster)[892451]: INFO: mssql_validate >Jul 14 17:08:38 ag(ag_cluster)[892451]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:08:38 ag(ag_cluster)[892451]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:08:38 ag(ag_cluster)[892451]: INFO: Resource agent invoked with: monitor >Jul 14 17:08:38 ag(ag_cluster)[892451]: INFO: mssql_monitor >Jul 14 17:08:38 ag(ag_cluster)[892451]: INFO: monitor: 2023/07/14 17:08:38 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:08:38 ag(ag_cluster)[892451]: INFO: monitor: 2023/07/14 17:08:38 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:08:38 ag(ag_cluster)[892451]: INFO: monitor: 2023/07/14 17:08:38 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:08:38 ag(ag_cluster)[892451]: INFO: monitor: 2023/07/14 17:08:38 Connected to the instance at localhost:1433 >Jul 14 17:08:43 ag(ag_cluster)[892451]: INFO: monitor: 2023/07/14 17:08:43 Monitor Caller is: monitor. >Jul 14 17:08:43 ag(ag_cluster)[892451]: INFO: monitor: 2023/07/14 17:08:43 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:08:43 ag(ag_cluster)[892451]: INFO: monitor: 2023/07/14 17:08:43 Replica is RESOLVING (0) >Jul 14 17:08:43 ag(ag_cluster)[892451]: INFO: monitor: 2023/07/14 17:08:43 Instance name is server3. >Jul 14 17:08:43 ag(ag_cluster)[892451]: INFO: monitor: 2023/07/14 17:08:43 Current master is server2. >Jul 14 17:08:43 ag(ag_cluster)[892451]: INFO: monitor: 2023/07/14 17:08:43 Setting the role to Secondary. >Jul 14 17:08:43 ag(ag_cluster)[892451]: INFO: monitor: 2023/07/14 17:08:43 Setting replica to SECONDARY role... >Jul 14 17:08:43 ag(ag_cluster)[892451]: INFO: monitor: 2023/07/14 17:08:43 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:08:43 ag(ag_cluster)[892451]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:08:43 ag(ag_cluster)[892451]: INFO: lease_expiry after monitor update: >Jul 14 17:08:43 ag(ag_cluster)[892451]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:08:43 ag(ag_cluster)[892451]: INFO: ag_cluster monitor : 0 >Jul 14 17:08:43 ag(ag_cluster)[892622]: INFO: mssql_validate >Jul 14 17:08:43 ag(ag_cluster)[892622]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:08:43 ag(ag_cluster)[892622]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:08:44 ag(ag_cluster)[892622]: INFO: Resource agent invoked with: monitor >Jul 14 17:08:44 ag(ag_cluster)[892622]: INFO: mssql_monitor >Jul 14 17:08:44 ag(ag_cluster)[892622]: INFO: monitor: 2023/07/14 17:08:44 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:08:44 ag(ag_cluster)[892622]: INFO: monitor: 2023/07/14 17:08:44 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:08:44 ag(ag_cluster)[892622]: INFO: monitor: 2023/07/14 17:08:44 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:08:44 ag(ag_cluster)[892622]: INFO: monitor: 2023/07/14 17:08:44 Connected to the instance at localhost:1433 >Jul 14 17:08:49 ag(ag_cluster)[892622]: INFO: monitor: 2023/07/14 17:08:49 Monitor Caller is: monitor. >Jul 14 17:08:49 ag(ag_cluster)[892622]: INFO: monitor: 2023/07/14 17:08:49 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:08:49 ag(ag_cluster)[892622]: INFO: monitor: 2023/07/14 17:08:49 Replica is RESOLVING (0) >Jul 14 17:08:49 ag(ag_cluster)[892622]: INFO: monitor: 2023/07/14 17:08:49 Instance name is server3. >Jul 14 17:08:49 ag(ag_cluster)[892622]: INFO: monitor: 2023/07/14 17:08:49 Current master is server2. >Jul 14 17:08:49 ag(ag_cluster)[892622]: INFO: monitor: 2023/07/14 17:08:49 Setting the role to Secondary. >Jul 14 17:08:49 ag(ag_cluster)[892622]: INFO: monitor: 2023/07/14 17:08:49 Setting replica to SECONDARY role... >Jul 14 17:08:49 ag(ag_cluster)[892622]: INFO: monitor: 2023/07/14 17:08:49 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:08:49 ag(ag_cluster)[892622]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:08:49 ag(ag_cluster)[892622]: INFO: lease_expiry after monitor update: >Jul 14 17:08:49 ag(ag_cluster)[892622]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:08:49 ag(ag_cluster)[892622]: INFO: ag_cluster monitor : 0 >Jul 14 17:08:56 ag(ag_cluster)[892795]: INFO: mssql_validate >Jul 14 17:08:56 ag(ag_cluster)[892795]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:08:56 ag(ag_cluster)[892795]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:08:56 ag(ag_cluster)[892795]: INFO: Resource agent invoked with: monitor >Jul 14 17:08:56 ag(ag_cluster)[892795]: INFO: mssql_monitor >Jul 14 17:08:56 ag(ag_cluster)[892795]: INFO: monitor: 2023/07/14 17:08:56 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:08:56 ag(ag_cluster)[892795]: INFO: monitor: 2023/07/14 17:08:56 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:08:56 ag(ag_cluster)[892795]: INFO: monitor: 2023/07/14 17:08:56 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:08:56 ag(ag_cluster)[892795]: INFO: monitor: 2023/07/14 17:08:56 Connected to the instance at localhost:1433 >Jul 14 17:09:01 ag(ag_cluster)[892795]: INFO: monitor: 2023/07/14 17:09:01 Monitor Caller is: monitor. >Jul 14 17:09:01 ag(ag_cluster)[892795]: INFO: monitor: 2023/07/14 17:09:01 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:09:01 ag(ag_cluster)[892795]: INFO: monitor: 2023/07/14 17:09:01 Replica is RESOLVING (0) >Jul 14 17:09:01 ag(ag_cluster)[892795]: INFO: monitor: 2023/07/14 17:09:01 Instance name is server3. >Jul 14 17:09:01 ag(ag_cluster)[892795]: INFO: monitor: 2023/07/14 17:09:01 Current master is server2. >Jul 14 17:09:01 ag(ag_cluster)[892795]: INFO: monitor: 2023/07/14 17:09:01 Setting the role to Secondary. >Jul 14 17:09:01 ag(ag_cluster)[892795]: INFO: monitor: 2023/07/14 17:09:01 Setting replica to SECONDARY role... >Jul 14 17:09:01 ag(ag_cluster)[892795]: INFO: monitor: 2023/07/14 17:09:01 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:09:01 ag(ag_cluster)[892795]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:09:01 ag(ag_cluster)[892795]: INFO: lease_expiry after monitor update: >Jul 14 17:09:01 ag(ag_cluster)[892795]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:09:01 ag(ag_cluster)[892795]: INFO: ag_cluster monitor : 0 >Jul 14 17:09:01 ag(ag_cluster)[892970]: INFO: mssql_validate >Jul 14 17:09:01 ag(ag_cluster)[892970]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:09:01 ag(ag_cluster)[892970]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:09:01 ag(ag_cluster)[892970]: INFO: Resource agent invoked with: monitor >Jul 14 17:09:01 ag(ag_cluster)[892970]: INFO: mssql_monitor >Jul 14 17:09:01 ag(ag_cluster)[892970]: INFO: monitor: 2023/07/14 17:09:01 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:09:01 ag(ag_cluster)[892970]: INFO: monitor: 2023/07/14 17:09:01 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:09:01 ag(ag_cluster)[892970]: INFO: monitor: 2023/07/14 17:09:01 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:09:01 ag(ag_cluster)[892970]: INFO: monitor: 2023/07/14 17:09:01 Connected to the instance at localhost:1433 >Jul 14 17:09:06 ag(ag_cluster)[892970]: INFO: monitor: 2023/07/14 17:09:06 Monitor Caller is: monitor. >Jul 14 17:09:06 ag(ag_cluster)[892970]: INFO: monitor: 2023/07/14 17:09:06 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:09:06 ag(ag_cluster)[892970]: INFO: monitor: 2023/07/14 17:09:06 Replica is RESOLVING (0) >Jul 14 17:09:06 ag(ag_cluster)[892970]: INFO: monitor: 2023/07/14 17:09:06 Instance name is server3. >Jul 14 17:09:06 ag(ag_cluster)[892970]: INFO: monitor: 2023/07/14 17:09:06 Current master is server2. >Jul 14 17:09:06 ag(ag_cluster)[892970]: INFO: monitor: 2023/07/14 17:09:06 Setting the role to Secondary. >Jul 14 17:09:06 ag(ag_cluster)[892970]: INFO: monitor: 2023/07/14 17:09:06 Setting replica to SECONDARY role... >Jul 14 17:09:06 ag(ag_cluster)[892970]: INFO: monitor: 2023/07/14 17:09:06 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:09:06 ag(ag_cluster)[892970]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:09:06 ag(ag_cluster)[892970]: INFO: lease_expiry after monitor update: >Jul 14 17:09:06 ag(ag_cluster)[892970]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:09:06 ag(ag_cluster)[892970]: INFO: ag_cluster monitor : 0 >Jul 14 17:09:07 ag(ag_cluster)[893142]: INFO: mssql_validate >Jul 14 17:09:07 ag(ag_cluster)[893142]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:09:07 ag(ag_cluster)[893142]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:09:07 ag(ag_cluster)[893142]: INFO: Resource agent invoked with: notify >Jul 14 17:09:07 ag(ag_cluster)[893142]: INFO: mssql_notify pre-start >Jul 14 17:09:07 ag(ag_cluster)[893142]: INFO: notify: 2023/07/14 17:09:07 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:09:07 ag(ag_cluster)[893142]: INFO: notify: 2023/07/14 17:09:07 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:09:07 ag(ag_cluster)[893142]: INFO: notify: 2023/07/14 17:09:07 Replica is RESOLVING (0) >Jul 14 17:09:07 ag(ag_cluster)[893142]: INFO: notify: 2023/07/14 17:09:07 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:09:07 ag(ag_cluster)[893142]: INFO: notify: 2023/07/14 17:09:07 Sequence number is 17F:F (1644972474383) >Jul 14 17:09:07 ag(ag_cluster)[893142]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:09:12 ag(ag_cluster)[893142]: INFO: ag_cluster notify : 0 >Jul 14 17:09:12 ag(ag_cluster)[893242]: INFO: mssql_validate >Jul 14 17:09:12 ag(ag_cluster)[893242]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:09:12 ag(ag_cluster)[893242]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:09:12 ag(ag_cluster)[893242]: INFO: Resource agent invoked with: notify >Jul 14 17:09:12 ag(ag_cluster)[893242]: INFO: mssql_notify pre-start >Jul 14 17:09:12 ag(ag_cluster)[893242]: INFO: notify: 2023/07/14 17:09:12 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:09:12 ag(ag_cluster)[893242]: INFO: notify: 2023/07/14 17:09:12 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:09:12 ag(ag_cluster)[893242]: INFO: notify: 2023/07/14 17:09:12 Replica is RESOLVING (0) >Jul 14 17:09:12 ag(ag_cluster)[893242]: INFO: notify: 2023/07/14 17:09:12 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:09:12 ag(ag_cluster)[893242]: INFO: notify: 2023/07/14 17:09:12 Sequence number is 17F:F (1644972474383) >Jul 14 17:09:12 ag(ag_cluster)[893242]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:09:17 ag(ag_cluster)[893242]: INFO: ag_cluster notify : 0 >Jul 14 17:09:17 ag(ag_cluster)[893347]: INFO: mssql_validate >Jul 14 17:09:17 ag(ag_cluster)[893347]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:09:17 ag(ag_cluster)[893347]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:09:17 ag(ag_cluster)[893347]: INFO: Resource agent invoked with: monitor >Jul 14 17:09:17 ag(ag_cluster)[893347]: INFO: mssql_monitor >Jul 14 17:09:17 ag(ag_cluster)[893347]: INFO: monitor: 2023/07/14 17:09:17 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:09:17 ag(ag_cluster)[893347]: INFO: monitor: 2023/07/14 17:09:17 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:09:17 ag(ag_cluster)[893347]: INFO: monitor: 2023/07/14 17:09:17 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:09:17 ag(ag_cluster)[893347]: INFO: monitor: 2023/07/14 17:09:17 Connected to the instance at localhost:1433 >Jul 14 17:09:22 ag(ag_cluster)[893347]: INFO: monitor: 2023/07/14 17:09:22 Monitor Caller is: monitor. >Jul 14 17:09:23 ag(ag_cluster)[893347]: INFO: monitor: 2023/07/14 17:09:22 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:09:23 ag(ag_cluster)[893347]: INFO: monitor: 2023/07/14 17:09:22 Replica is RESOLVING (0) >Jul 14 17:09:23 ag(ag_cluster)[893347]: INFO: monitor: 2023/07/14 17:09:22 Instance name is server3. >Jul 14 17:09:23 ag(ag_cluster)[893347]: INFO: monitor: 2023/07/14 17:09:22 Current master is server2. >Jul 14 17:09:23 ag(ag_cluster)[893347]: INFO: monitor: 2023/07/14 17:09:22 Setting the role to Secondary. >Jul 14 17:09:23 ag(ag_cluster)[893347]: INFO: monitor: 2023/07/14 17:09:22 Setting replica to SECONDARY role... >Jul 14 17:09:23 ag(ag_cluster)[893347]: INFO: monitor: 2023/07/14 17:09:22 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:09:23 ag(ag_cluster)[893347]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:09:23 ag(ag_cluster)[893347]: INFO: lease_expiry after monitor update: >Jul 14 17:09:23 ag(ag_cluster)[893347]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:09:23 ag(ag_cluster)[893347]: INFO: ag_cluster monitor : 0 >Jul 14 17:09:23 ag(ag_cluster)[893520]: INFO: mssql_validate >Jul 14 17:09:23 ag(ag_cluster)[893520]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:09:23 ag(ag_cluster)[893520]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:09:23 ag(ag_cluster)[893520]: INFO: Resource agent invoked with: monitor >Jul 14 17:09:23 ag(ag_cluster)[893520]: INFO: mssql_monitor >Jul 14 17:09:23 ag(ag_cluster)[893520]: INFO: monitor: 2023/07/14 17:09:23 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:09:23 ag(ag_cluster)[893520]: INFO: monitor: 2023/07/14 17:09:23 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:09:23 ag(ag_cluster)[893520]: INFO: monitor: 2023/07/14 17:09:23 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:09:23 ag(ag_cluster)[893520]: INFO: monitor: 2023/07/14 17:09:23 Connected to the instance at localhost:1433 >Jul 14 17:09:28 ag(ag_cluster)[893520]: INFO: monitor: 2023/07/14 17:09:28 Monitor Caller is: monitor. >Jul 14 17:09:28 ag(ag_cluster)[893520]: INFO: monitor: 2023/07/14 17:09:28 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:09:28 ag(ag_cluster)[893520]: INFO: monitor: 2023/07/14 17:09:28 Replica is RESOLVING (0) >Jul 14 17:09:28 ag(ag_cluster)[893520]: INFO: monitor: 2023/07/14 17:09:28 Instance name is server3. >Jul 14 17:09:28 ag(ag_cluster)[893520]: INFO: monitor: 2023/07/14 17:09:28 Current master is server2. >Jul 14 17:09:28 ag(ag_cluster)[893520]: INFO: monitor: 2023/07/14 17:09:28 Setting the role to Secondary. >Jul 14 17:09:28 ag(ag_cluster)[893520]: INFO: monitor: 2023/07/14 17:09:28 Setting replica to SECONDARY role... >Jul 14 17:09:28 ag(ag_cluster)[893520]: INFO: monitor: 2023/07/14 17:09:28 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:09:28 ag(ag_cluster)[893520]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:09:28 ag(ag_cluster)[893520]: INFO: lease_expiry after monitor update: >Jul 14 17:09:28 ag(ag_cluster)[893520]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:09:28 ag(ag_cluster)[893520]: INFO: ag_cluster monitor : 0 >Jul 14 17:09:28 ag(ag_cluster)[893692]: INFO: mssql_validate >Jul 14 17:09:28 ag(ag_cluster)[893692]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:09:28 ag(ag_cluster)[893692]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:09:28 ag(ag_cluster)[893692]: INFO: Resource agent invoked with: notify >Jul 14 17:09:28 ag(ag_cluster)[893692]: INFO: mssql_notify post-start >Jul 14 17:09:28 ag(ag_cluster)[893692]: INFO: ag_cluster notify : 0 >Jul 14 17:09:28 ag(ag_cluster)[893745]: INFO: mssql_validate >Jul 14 17:09:28 ag(ag_cluster)[893745]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:09:28 ag(ag_cluster)[893745]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:09:28 ag(ag_cluster)[893745]: INFO: Resource agent invoked with: notify >Jul 14 17:09:28 ag(ag_cluster)[893745]: INFO: mssql_notify pre-stop >Jul 14 17:09:28 ag(ag_cluster)[893745]: INFO: ag_cluster notify : 0 >Jul 14 17:09:29 ag(ag_cluster)[893792]: INFO: mssql_validate >Jul 14 17:09:29 ag(ag_cluster)[893792]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:09:29 ag(ag_cluster)[893792]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:09:29 ag(ag_cluster)[893792]: INFO: Resource agent invoked with: notify >Jul 14 17:09:29 ag(ag_cluster)[893792]: INFO: mssql_notify post-stop >Jul 14 17:09:29 ag(ag_cluster)[893792]: INFO: notify: 2023/07/14 17:09:29 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-stop]; connection-timeout [30]; health-threshold [3]; action [post-stop] >Jul 14 17:09:29 ag(ag_cluster)[893792]: INFO: notify: 2023/07/14 17:09:29 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:09:29 ag(ag_cluster)[893792]: INFO: notify: 2023/07/14 17:09:29 Replica is RESOLVING (0) >Jul 14 17:09:29 ag(ag_cluster)[893792]: INFO: ag_cluster notify : 0 >Jul 14 17:09:35 ag(ag_cluster)[893867]: INFO: mssql_validate >Jul 14 17:09:35 ag(ag_cluster)[893867]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:09:35 ag(ag_cluster)[893867]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:09:35 ag(ag_cluster)[893867]: INFO: Resource agent invoked with: monitor >Jul 14 17:09:35 ag(ag_cluster)[893867]: INFO: mssql_monitor >Jul 14 17:09:35 ag(ag_cluster)[893867]: INFO: monitor: 2023/07/14 17:09:35 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:09:35 ag(ag_cluster)[893867]: INFO: monitor: 2023/07/14 17:09:35 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:09:35 ag(ag_cluster)[893867]: INFO: monitor: 2023/07/14 17:09:35 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:09:35 ag(ag_cluster)[893867]: INFO: monitor: 2023/07/14 17:09:35 Connected to the instance at localhost:1433 >Jul 14 17:09:40 ag(ag_cluster)[893867]: INFO: monitor: 2023/07/14 17:09:40 Monitor Caller is: monitor. >Jul 14 17:09:40 ag(ag_cluster)[893867]: INFO: monitor: 2023/07/14 17:09:40 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:09:40 ag(ag_cluster)[893867]: INFO: monitor: 2023/07/14 17:09:40 Replica is RESOLVING (0) >Jul 14 17:09:40 ag(ag_cluster)[893867]: INFO: monitor: 2023/07/14 17:09:40 Instance name is server3. >Jul 14 17:09:40 ag(ag_cluster)[893867]: INFO: monitor: 2023/07/14 17:09:40 Current master is server2. >Jul 14 17:09:40 ag(ag_cluster)[893867]: INFO: monitor: 2023/07/14 17:09:40 Setting the role to Secondary. >Jul 14 17:09:40 ag(ag_cluster)[893867]: INFO: monitor: 2023/07/14 17:09:40 Setting replica to SECONDARY role... >Jul 14 17:09:40 ag(ag_cluster)[893867]: INFO: monitor: 2023/07/14 17:09:40 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:09:40 ag(ag_cluster)[893867]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:09:40 ag(ag_cluster)[893867]: INFO: lease_expiry after monitor update: >Jul 14 17:09:40 ag(ag_cluster)[893867]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:09:40 ag(ag_cluster)[893867]: INFO: ag_cluster monitor : 0 >Jul 14 17:09:40 ag(ag_cluster)[894040]: INFO: mssql_validate >Jul 14 17:09:40 ag(ag_cluster)[894040]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:09:40 ag(ag_cluster)[894040]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:09:40 ag(ag_cluster)[894040]: INFO: Resource agent invoked with: monitor >Jul 14 17:09:40 ag(ag_cluster)[894040]: INFO: mssql_monitor >Jul 14 17:09:40 ag(ag_cluster)[894040]: INFO: monitor: 2023/07/14 17:09:40 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:09:40 ag(ag_cluster)[894040]: INFO: monitor: 2023/07/14 17:09:40 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:09:40 ag(ag_cluster)[894040]: INFO: monitor: 2023/07/14 17:09:40 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:09:40 ag(ag_cluster)[894040]: INFO: monitor: 2023/07/14 17:09:40 Connected to the instance at localhost:1433 >Jul 14 17:09:45 ag(ag_cluster)[894040]: INFO: monitor: 2023/07/14 17:09:45 Monitor Caller is: monitor. >Jul 14 17:09:45 ag(ag_cluster)[894040]: INFO: monitor: 2023/07/14 17:09:45 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:09:45 ag(ag_cluster)[894040]: INFO: monitor: 2023/07/14 17:09:45 Replica is RESOLVING (0) >Jul 14 17:09:46 ag(ag_cluster)[894040]: INFO: monitor: 2023/07/14 17:09:45 Instance name is server3. >Jul 14 17:09:46 ag(ag_cluster)[894040]: INFO: monitor: 2023/07/14 17:09:45 Current master is server2. >Jul 14 17:09:46 ag(ag_cluster)[894040]: INFO: monitor: 2023/07/14 17:09:45 Setting the role to Secondary. >Jul 14 17:09:46 ag(ag_cluster)[894040]: INFO: monitor: 2023/07/14 17:09:45 Setting replica to SECONDARY role... >Jul 14 17:09:46 ag(ag_cluster)[894040]: INFO: monitor: 2023/07/14 17:09:45 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:09:46 ag(ag_cluster)[894040]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:09:46 ag(ag_cluster)[894040]: INFO: lease_expiry after monitor update: >Jul 14 17:09:46 ag(ag_cluster)[894040]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:09:46 ag(ag_cluster)[894040]: INFO: ag_cluster monitor : 0 >Jul 14 17:09:52 ag(ag_cluster)[894211]: INFO: mssql_validate >Jul 14 17:09:52 ag(ag_cluster)[894211]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:09:52 ag(ag_cluster)[894211]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:09:52 ag(ag_cluster)[894211]: INFO: Resource agent invoked with: monitor >Jul 14 17:09:52 ag(ag_cluster)[894211]: INFO: mssql_monitor >Jul 14 17:09:52 ag(ag_cluster)[894211]: INFO: monitor: 2023/07/14 17:09:52 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:09:52 ag(ag_cluster)[894211]: INFO: monitor: 2023/07/14 17:09:52 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:09:52 ag(ag_cluster)[894211]: INFO: monitor: 2023/07/14 17:09:52 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:09:52 ag(ag_cluster)[894211]: INFO: monitor: 2023/07/14 17:09:52 Connected to the instance at localhost:1433 >Jul 14 17:09:57 ag(ag_cluster)[894211]: INFO: monitor: 2023/07/14 17:09:57 Monitor Caller is: monitor. >Jul 14 17:09:57 ag(ag_cluster)[894211]: INFO: monitor: 2023/07/14 17:09:57 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:09:58 ag(ag_cluster)[894211]: INFO: monitor: 2023/07/14 17:09:57 Replica is RESOLVING (0) >Jul 14 17:09:58 ag(ag_cluster)[894211]: INFO: monitor: 2023/07/14 17:09:57 Instance name is server3. >Jul 14 17:09:58 ag(ag_cluster)[894211]: INFO: monitor: 2023/07/14 17:09:57 Current master is server2. >Jul 14 17:09:58 ag(ag_cluster)[894211]: INFO: monitor: 2023/07/14 17:09:57 Setting the role to Secondary. >Jul 14 17:09:58 ag(ag_cluster)[894211]: INFO: monitor: 2023/07/14 17:09:57 Setting replica to SECONDARY role... >Jul 14 17:09:58 ag(ag_cluster)[894211]: INFO: monitor: 2023/07/14 17:09:57 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:09:58 ag(ag_cluster)[894211]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:09:58 ag(ag_cluster)[894211]: INFO: lease_expiry after monitor update: >Jul 14 17:09:58 ag(ag_cluster)[894211]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:09:58 ag(ag_cluster)[894211]: INFO: ag_cluster monitor : 0 >Jul 14 17:09:58 ag(ag_cluster)[894383]: INFO: mssql_validate >Jul 14 17:09:58 ag(ag_cluster)[894383]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:09:58 ag(ag_cluster)[894383]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:09:58 ag(ag_cluster)[894383]: INFO: Resource agent invoked with: monitor >Jul 14 17:09:58 ag(ag_cluster)[894383]: INFO: mssql_monitor >Jul 14 17:09:58 ag(ag_cluster)[894383]: INFO: monitor: 2023/07/14 17:09:58 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:09:58 ag(ag_cluster)[894383]: INFO: monitor: 2023/07/14 17:09:58 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:09:58 ag(ag_cluster)[894383]: INFO: monitor: 2023/07/14 17:09:58 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:09:58 ag(ag_cluster)[894383]: INFO: monitor: 2023/07/14 17:09:58 Connected to the instance at localhost:1433 >Jul 14 17:10:03 ag(ag_cluster)[894383]: INFO: monitor: 2023/07/14 17:10:03 Monitor Caller is: monitor. >Jul 14 17:10:03 ag(ag_cluster)[894383]: INFO: monitor: 2023/07/14 17:10:03 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:10:03 ag(ag_cluster)[894383]: INFO: monitor: 2023/07/14 17:10:03 Replica is RESOLVING (0) >Jul 14 17:10:03 ag(ag_cluster)[894383]: INFO: monitor: 2023/07/14 17:10:03 Instance name is server3. >Jul 14 17:10:03 ag(ag_cluster)[894383]: INFO: monitor: 2023/07/14 17:10:03 Current master is server2. >Jul 14 17:10:03 ag(ag_cluster)[894383]: INFO: monitor: 2023/07/14 17:10:03 Setting the role to Secondary. >Jul 14 17:10:03 ag(ag_cluster)[894383]: INFO: monitor: 2023/07/14 17:10:03 Setting replica to SECONDARY role... >Jul 14 17:10:03 ag(ag_cluster)[894383]: INFO: monitor: 2023/07/14 17:10:03 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:10:03 ag(ag_cluster)[894383]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:10:03 ag(ag_cluster)[894383]: INFO: lease_expiry after monitor update: >Jul 14 17:10:03 ag(ag_cluster)[894383]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:10:03 ag(ag_cluster)[894383]: INFO: ag_cluster monitor : 0 >Jul 14 17:10:07 ag(ag_cluster)[894556]: INFO: mssql_validate >Jul 14 17:10:07 ag(ag_cluster)[894556]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:10:07 ag(ag_cluster)[894556]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:10:07 ag(ag_cluster)[894556]: INFO: Resource agent invoked with: notify >Jul 14 17:10:07 ag(ag_cluster)[894556]: INFO: mssql_notify pre-demote >Jul 14 17:10:07 ag(ag_cluster)[894556]: INFO: ag_cluster notify : 0 >Jul 14 17:10:08 ag(ag_cluster)[894602]: INFO: mssql_validate >Jul 14 17:10:08 ag(ag_cluster)[894602]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:10:08 ag(ag_cluster)[894602]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:10:08 ag(ag_cluster)[894602]: INFO: Resource agent invoked with: notify >Jul 14 17:10:08 ag(ag_cluster)[894602]: INFO: mssql_notify post-demote >Jul 14 17:10:08 ag(ag_cluster)[894602]: INFO: ag_cluster notify : 0 >Jul 14 17:10:08 ag(ag_cluster)[894647]: INFO: mssql_validate >Jul 14 17:10:08 ag(ag_cluster)[894647]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:10:08 ag(ag_cluster)[894647]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:10:08 ag(ag_cluster)[894647]: INFO: Resource agent invoked with: notify >Jul 14 17:10:08 ag(ag_cluster)[894647]: INFO: mssql_notify pre-stop >Jul 14 17:10:08 ag(ag_cluster)[894647]: INFO: ag_cluster notify : 0 >Jul 14 17:10:08 ag(ag_cluster)[894692]: INFO: mssql_validate >Jul 14 17:10:08 ag(ag_cluster)[894692]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:10:08 ag(ag_cluster)[894692]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:10:08 ag(ag_cluster)[894692]: INFO: Resource agent invoked with: notify >Jul 14 17:10:08 ag(ag_cluster)[894692]: INFO: mssql_notify post-stop >Jul 14 17:10:08 ag(ag_cluster)[894692]: INFO: notify: 2023/07/14 17:10:08 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-stop]; connection-timeout [30]; health-threshold [3]; action [post-stop] >Jul 14 17:10:08 ag(ag_cluster)[894692]: INFO: notify: 2023/07/14 17:10:08 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:10:08 ag(ag_cluster)[894692]: INFO: notify: 2023/07/14 17:10:08 Replica is RESOLVING (0) >Jul 14 17:10:08 ag(ag_cluster)[894692]: INFO: ag_cluster notify : 0 >Jul 14 17:10:09 ag(ag_cluster)[894768]: INFO: mssql_validate >Jul 14 17:10:09 ag(ag_cluster)[894768]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:10:09 ag(ag_cluster)[894768]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:10:09 ag(ag_cluster)[894768]: INFO: Resource agent invoked with: notify >Jul 14 17:10:09 ag(ag_cluster)[894768]: INFO: mssql_notify pre-start >Jul 14 17:10:09 ag(ag_cluster)[894768]: INFO: notify: 2023/07/14 17:10:09 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:10:09 ag(ag_cluster)[894768]: INFO: notify: 2023/07/14 17:10:09 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:10:09 ag(ag_cluster)[894768]: INFO: notify: 2023/07/14 17:10:09 Replica is RESOLVING (0) >Jul 14 17:10:09 ag(ag_cluster)[894768]: INFO: notify: 2023/07/14 17:10:09 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:10:09 ag(ag_cluster)[894768]: INFO: notify: 2023/07/14 17:10:09 Sequence number is 17F:F (1644972474383) >Jul 14 17:10:09 ag(ag_cluster)[894768]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:10:14 ag(ag_cluster)[894768]: INFO: ag_cluster notify : 0 >Jul 14 17:10:14 ag(ag_cluster)[894867]: INFO: mssql_validate >Jul 14 17:10:14 ag(ag_cluster)[894867]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:10:14 ag(ag_cluster)[894867]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:10:14 ag(ag_cluster)[894867]: INFO: Resource agent invoked with: monitor >Jul 14 17:10:14 ag(ag_cluster)[894867]: INFO: mssql_monitor >Jul 14 17:10:14 ag(ag_cluster)[894867]: INFO: monitor: 2023/07/14 17:10:14 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:10:14 ag(ag_cluster)[894867]: INFO: monitor: 2023/07/14 17:10:14 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master []; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:10:14 ag(ag_cluster)[894867]: INFO: monitor: 2023/07/14 17:10:14 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:10:14 ag(ag_cluster)[894867]: INFO: monitor: 2023/07/14 17:10:14 Connected to the instance at localhost:1433 >Jul 14 17:10:19 ag(ag_cluster)[894867]: INFO: monitor: 2023/07/14 17:10:19 Monitor Caller is: monitor. >Jul 14 17:10:19 ag(ag_cluster)[894867]: INFO: monitor: 2023/07/14 17:10:19 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:10:19 ag(ag_cluster)[894867]: INFO: monitor: 2023/07/14 17:10:19 Replica is RESOLVING (0) >Jul 14 17:10:19 ag(ag_cluster)[894867]: INFO: monitor: 2023/07/14 17:10:19 Instance name is server3. >Jul 14 17:10:19 ag(ag_cluster)[894867]: INFO: monitor: 2023/07/14 17:10:19 Current master is . >Jul 14 17:10:19 ag(ag_cluster)[894867]: INFO: monitor: PROMOTION_SCORE: 10 >Jul 14 17:10:19 ag(ag_cluster)[894867]: INFO: lease_expiry after monitor update: >Jul 14 17:10:19 ag(ag_cluster)[894867]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:10:19 ag(ag_cluster)[894867]: INFO: ag_cluster monitor : 0 >Jul 14 17:10:19 ag(ag_cluster)[895026]: INFO: mssql_validate >Jul 14 17:10:19 ag(ag_cluster)[895026]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:10:19 ag(ag_cluster)[895026]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:10:19 ag(ag_cluster)[895026]: INFO: Resource agent invoked with: monitor >Jul 14 17:10:19 ag(ag_cluster)[895026]: INFO: mssql_monitor >Jul 14 17:10:19 ag(ag_cluster)[895026]: INFO: monitor: 2023/07/14 17:10:19 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:10:19 ag(ag_cluster)[895026]: INFO: monitor: 2023/07/14 17:10:19 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master []; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:10:19 ag(ag_cluster)[895026]: INFO: monitor: 2023/07/14 17:10:19 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:10:19 ag(ag_cluster)[895026]: INFO: monitor: 2023/07/14 17:10:19 Connected to the instance at localhost:1433 >Jul 14 17:10:24 ag(ag_cluster)[895026]: INFO: monitor: 2023/07/14 17:10:24 Monitor Caller is: monitor. >Jul 14 17:10:24 ag(ag_cluster)[895026]: INFO: monitor: 2023/07/14 17:10:24 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:10:24 ag(ag_cluster)[895026]: INFO: monitor: 2023/07/14 17:10:24 Replica is RESOLVING (0) >Jul 14 17:10:24 ag(ag_cluster)[895026]: INFO: monitor: 2023/07/14 17:10:24 Instance name is server3. >Jul 14 17:10:24 ag(ag_cluster)[895026]: INFO: monitor: 2023/07/14 17:10:24 Current master is . >Jul 14 17:10:24 ag(ag_cluster)[895026]: INFO: monitor: PROMOTION_SCORE: 10 >Jul 14 17:10:24 ag(ag_cluster)[895026]: INFO: lease_expiry after monitor update: >Jul 14 17:10:24 ag(ag_cluster)[895026]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:10:24 ag(ag_cluster)[895026]: INFO: ag_cluster monitor : 0 >Jul 14 17:10:25 ag(ag_cluster)[895181]: INFO: mssql_validate >Jul 14 17:10:25 ag(ag_cluster)[895181]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:10:25 ag(ag_cluster)[895181]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:10:25 ag(ag_cluster)[895181]: INFO: Resource agent invoked with: notify >Jul 14 17:10:25 ag(ag_cluster)[895181]: INFO: mssql_notify pre-start >Jul 14 17:10:25 ag(ag_cluster)[895181]: INFO: notify: 2023/07/14 17:10:25 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:10:25 ag(ag_cluster)[895181]: INFO: notify: 2023/07/14 17:10:25 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:10:25 ag(ag_cluster)[895181]: INFO: notify: 2023/07/14 17:10:25 Replica is RESOLVING (0) >Jul 14 17:10:25 ag(ag_cluster)[895181]: INFO: notify: 2023/07/14 17:10:25 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:10:25 ag(ag_cluster)[895181]: INFO: notify: 2023/07/14 17:10:25 Sequence number is 17F:F (1644972474383) >Jul 14 17:10:25 ag(ag_cluster)[895181]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:10:30 ag(ag_cluster)[895181]: INFO: ag_cluster notify : 0 >Jul 14 17:10:30 ag(ag_cluster)[895288]: INFO: mssql_validate >Jul 14 17:10:30 ag(ag_cluster)[895288]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:10:30 ag(ag_cluster)[895288]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:10:30 ag(ag_cluster)[895288]: INFO: Resource agent invoked with: notify >Jul 14 17:10:30 ag(ag_cluster)[895288]: INFO: mssql_notify pre-start >Jul 14 17:10:30 ag(ag_cluster)[895288]: INFO: notify: 2023/07/14 17:10:30 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:10:30 ag(ag_cluster)[895288]: INFO: notify: 2023/07/14 17:10:30 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:10:30 ag(ag_cluster)[895288]: INFO: notify: 2023/07/14 17:10:30 Replica is RESOLVING (0) >Jul 14 17:10:30 ag(ag_cluster)[895288]: INFO: notify: 2023/07/14 17:10:30 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:10:30 ag(ag_cluster)[895288]: INFO: notify: 2023/07/14 17:10:30 Sequence number is 17F:F (1644972474383) >Jul 14 17:10:30 ag(ag_cluster)[895288]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:10:35 ag(ag_cluster)[895288]: INFO: ag_cluster notify : 0 >Jul 14 17:10:35 ag(ag_cluster)[895387]: INFO: mssql_validate >Jul 14 17:10:35 ag(ag_cluster)[895387]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:10:35 ag(ag_cluster)[895387]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:10:35 ag(ag_cluster)[895387]: INFO: Resource agent invoked with: notify >Jul 14 17:10:35 ag(ag_cluster)[895387]: INFO: mssql_notify pre-start >Jul 14 17:10:35 ag(ag_cluster)[895387]: INFO: notify: 2023/07/14 17:10:35 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:10:35 ag(ag_cluster)[895387]: INFO: notify: 2023/07/14 17:10:35 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:10:35 ag(ag_cluster)[895387]: INFO: notify: 2023/07/14 17:10:35 Replica is RESOLVING (0) >Jul 14 17:10:35 ag(ag_cluster)[895387]: INFO: notify: 2023/07/14 17:10:35 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:10:35 ag(ag_cluster)[895387]: INFO: notify: 2023/07/14 17:10:35 Sequence number is 17F:F (1644972474383) >Jul 14 17:10:35 ag(ag_cluster)[895387]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:10:40 ag(ag_cluster)[895387]: INFO: ag_cluster notify : 0 >Jul 14 17:10:46 ag(ag_cluster)[895487]: INFO: mssql_validate >Jul 14 17:10:46 ag(ag_cluster)[895487]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:10:46 ag(ag_cluster)[895487]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:10:46 ag(ag_cluster)[895487]: INFO: Resource agent invoked with: notify >Jul 14 17:10:46 ag(ag_cluster)[895487]: INFO: mssql_notify post-start >Jul 14 17:10:46 ag(ag_cluster)[895487]: INFO: ag_cluster notify : 0 >Jul 14 17:10:46 ag(ag_cluster)[895533]: INFO: mssql_validate >Jul 14 17:10:46 ag(ag_cluster)[895533]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:10:46 ag(ag_cluster)[895533]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:10:46 ag(ag_cluster)[895533]: INFO: Resource agent invoked with: notify >Jul 14 17:10:46 ag(ag_cluster)[895533]: INFO: mssql_notify pre-stop >Jul 14 17:10:46 ag(ag_cluster)[895533]: INFO: ag_cluster notify : 0 >Jul 14 17:10:46 ag(ag_cluster)[895578]: INFO: mssql_validate >Jul 14 17:10:46 ag(ag_cluster)[895578]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:10:46 ag(ag_cluster)[895578]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:10:46 ag(ag_cluster)[895578]: INFO: Resource agent invoked with: notify >Jul 14 17:10:46 ag(ag_cluster)[895578]: INFO: mssql_notify post-stop >Jul 14 17:10:46 ag(ag_cluster)[895578]: INFO: notify: 2023/07/14 17:10:46 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-stop]; connection-timeout [30]; health-threshold [3]; action [post-stop] >Jul 14 17:10:46 ag(ag_cluster)[895578]: INFO: notify: 2023/07/14 17:10:46 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:10:46 ag(ag_cluster)[895578]: INFO: notify: 2023/07/14 17:10:46 Replica is RESOLVING (0) >Jul 14 17:10:46 ag(ag_cluster)[895578]: INFO: ag_cluster notify : 0 >Jul 14 17:10:46 ag(ag_cluster)[895652]: INFO: mssql_validate >Jul 14 17:10:46 ag(ag_cluster)[895652]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:10:46 ag(ag_cluster)[895652]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:10:46 ag(ag_cluster)[895652]: INFO: Resource agent invoked with: notify >Jul 14 17:10:46 ag(ag_cluster)[895652]: INFO: mssql_notify pre-promote >Jul 14 17:10:46 ag(ag_cluster)[895652]: INFO: notify: 2023/07/14 17:10:46 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-promote]; connection-timeout [30]; health-threshold [3]; action [pre-promote] >Jul 14 17:10:46 ag(ag_cluster)[895652]: INFO: notify: 2023/07/14 17:10:46 From RetryExecute - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:10:46 ag(ag_cluster)[895652]: INFO: notify: 2023/07/14 17:10:46 Connected to the instance at localhost:1433 >Jul 14 17:10:51 ag(ag_cluster)[895652]: INFO: notify: 2023/07/14 17:10:51 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:10:51 ag(ag_cluster)[895652]: INFO: notify: 2023/07/14 17:10:51 Sequence number is 17F:F (1644972474383) >Jul 14 17:10:51 ag(ag_cluster)[895652]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:10:57 ag(ag_cluster)[895652]: INFO: ag_cluster notify : 0 >Jul 14 17:10:57 ag(ag_cluster)[895756]: INFO: mssql_validate >Jul 14 17:10:57 ag(ag_cluster)[895756]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:10:57 ag(ag_cluster)[895756]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:10:57 ag(ag_cluster)[895756]: INFO: Resource agent invoked with: notify >Jul 14 17:10:57 ag(ag_cluster)[895756]: INFO: mssql_notify pre-promote >Jul 14 17:10:57 ag(ag_cluster)[895756]: INFO: notify: 2023/07/14 17:10:57 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-promote]; connection-timeout [30]; health-threshold [3]; action [pre-promote] >Jul 14 17:10:57 ag(ag_cluster)[895756]: INFO: notify: 2023/07/14 17:10:57 From RetryExecute - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:10:57 ag(ag_cluster)[895756]: INFO: notify: 2023/07/14 17:10:57 Connected to the instance at localhost:1433 >Jul 14 17:11:02 ag(ag_cluster)[895756]: INFO: notify: 2023/07/14 17:11:02 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:11:02 ag(ag_cluster)[895756]: INFO: notify: 2023/07/14 17:11:02 Sequence number is 17F:F (1644972474383) >Jul 14 17:11:02 ag(ag_cluster)[895756]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:11:07 ag(ag_cluster)[895756]: INFO: ag_cluster notify : 0 >Jul 14 17:11:12 ag(ag_cluster)[895861]: INFO: mssql_validate >Jul 14 17:11:12 ag(ag_cluster)[895861]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:11:12 ag(ag_cluster)[895861]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:11:12 ag(ag_cluster)[895861]: INFO: Resource agent invoked with: notify >Jul 14 17:11:12 ag(ag_cluster)[895861]: INFO: mssql_notify post-promote >Jul 14 17:11:12 ag(ag_cluster)[895861]: INFO: notify: 2023/07/14 17:11:12 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-promote]; connection-timeout [30]; health-threshold [3]; action [post-promote] >Jul 14 17:11:13 ag(ag_cluster)[895861]: INFO: notify: 2023/07/14 17:11:13 [DEBUG] AG Helper PostPromote Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:11:13 ag(ag_cluster)[895861]: INFO: notify: 2023/07/14 17:11:13 Replica is RESOLVING (0) >Jul 14 17:11:13 ag(ag_cluster)[895861]: INFO: ag_cluster notify : 0 >Jul 14 17:11:13 ag(ag_cluster)[895937]: INFO: mssql_validate >Jul 14 17:11:13 ag(ag_cluster)[895937]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:11:13 ag(ag_cluster)[895937]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:11:13 ag(ag_cluster)[895937]: INFO: Resource agent invoked with: monitor >Jul 14 17:11:13 ag(ag_cluster)[895937]: INFO: mssql_monitor >Jul 14 17:11:13 ag(ag_cluster)[895937]: INFO: monitor: 2023/07/14 17:11:13 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:11:13 ag(ag_cluster)[895937]: INFO: monitor: 2023/07/14 17:11:13 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:11:13 ag(ag_cluster)[895937]: INFO: monitor: 2023/07/14 17:11:13 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:11:13 ag(ag_cluster)[895937]: INFO: monitor: 2023/07/14 17:11:13 Connected to the instance at localhost:1433 >Jul 14 17:11:18 ag(ag_cluster)[895937]: INFO: monitor: 2023/07/14 17:11:18 Monitor Caller is: monitor. >Jul 14 17:11:18 ag(ag_cluster)[895937]: INFO: monitor: 2023/07/14 17:11:18 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:11:18 ag(ag_cluster)[895937]: INFO: monitor: 2023/07/14 17:11:18 Replica is RESOLVING (0) >Jul 14 17:11:18 ag(ag_cluster)[895937]: INFO: monitor: 2023/07/14 17:11:18 Instance name is server3. >Jul 14 17:11:18 ag(ag_cluster)[895937]: INFO: monitor: 2023/07/14 17:11:18 Current master is server2. >Jul 14 17:11:18 ag(ag_cluster)[895937]: INFO: monitor: 2023/07/14 17:11:18 Setting the role to Secondary. >Jul 14 17:11:18 ag(ag_cluster)[895937]: INFO: monitor: 2023/07/14 17:11:18 Setting replica to SECONDARY role... >Jul 14 17:11:18 ag(ag_cluster)[895937]: INFO: monitor: 2023/07/14 17:11:18 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:11:18 ag(ag_cluster)[895937]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:11:18 ag(ag_cluster)[895937]: INFO: lease_expiry after monitor update: >Jul 14 17:11:18 ag(ag_cluster)[895937]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:11:18 ag(ag_cluster)[895937]: INFO: ag_cluster monitor : 0 >Jul 14 17:11:18 ag(ag_cluster)[896115]: INFO: mssql_validate >Jul 14 17:11:18 ag(ag_cluster)[896115]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:11:18 ag(ag_cluster)[896115]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:11:18 ag(ag_cluster)[896115]: INFO: Resource agent invoked with: monitor >Jul 14 17:11:18 ag(ag_cluster)[896115]: INFO: mssql_monitor >Jul 14 17:11:18 ag(ag_cluster)[896115]: INFO: monitor: 2023/07/14 17:11:18 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:11:18 ag(ag_cluster)[896115]: INFO: monitor: 2023/07/14 17:11:18 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:11:18 ag(ag_cluster)[896115]: INFO: monitor: 2023/07/14 17:11:18 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:11:18 ag(ag_cluster)[896115]: INFO: monitor: 2023/07/14 17:11:18 Connected to the instance at localhost:1433 >Jul 14 17:11:23 ag(ag_cluster)[896115]: INFO: monitor: 2023/07/14 17:11:23 Monitor Caller is: monitor. >Jul 14 17:11:23 ag(ag_cluster)[896115]: INFO: monitor: 2023/07/14 17:11:23 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:11:23 ag(ag_cluster)[896115]: INFO: monitor: 2023/07/14 17:11:23 Replica is RESOLVING (0) >Jul 14 17:11:23 ag(ag_cluster)[896115]: INFO: monitor: 2023/07/14 17:11:23 Instance name is server3. >Jul 14 17:11:23 ag(ag_cluster)[896115]: INFO: monitor: 2023/07/14 17:11:23 Current master is server2. >Jul 14 17:11:23 ag(ag_cluster)[896115]: INFO: monitor: 2023/07/14 17:11:23 Setting the role to Secondary. >Jul 14 17:11:23 ag(ag_cluster)[896115]: INFO: monitor: 2023/07/14 17:11:23 Setting replica to SECONDARY role... >Jul 14 17:11:23 ag(ag_cluster)[896115]: INFO: monitor: 2023/07/14 17:11:23 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:11:23 ag(ag_cluster)[896115]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:11:23 ag(ag_cluster)[896115]: INFO: lease_expiry after monitor update: >Jul 14 17:11:23 ag(ag_cluster)[896115]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:11:23 ag(ag_cluster)[896115]: INFO: ag_cluster monitor : 0 >Jul 14 17:11:30 ag(ag_cluster)[896298]: INFO: mssql_validate >Jul 14 17:11:30 ag(ag_cluster)[896298]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:11:30 ag(ag_cluster)[896298]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:11:30 ag(ag_cluster)[896298]: INFO: Resource agent invoked with: monitor >Jul 14 17:11:30 ag(ag_cluster)[896298]: INFO: mssql_monitor >Jul 14 17:11:30 ag(ag_cluster)[896298]: INFO: monitor: 2023/07/14 17:11:30 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:11:30 ag(ag_cluster)[896298]: INFO: monitor: 2023/07/14 17:11:30 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:11:30 ag(ag_cluster)[896298]: INFO: monitor: 2023/07/14 17:11:30 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:11:30 ag(ag_cluster)[896298]: INFO: monitor: 2023/07/14 17:11:30 Connected to the instance at localhost:1433 >Jul 14 17:11:35 ag(ag_cluster)[896298]: INFO: monitor: 2023/07/14 17:11:35 Monitor Caller is: monitor. >Jul 14 17:11:35 ag(ag_cluster)[896298]: INFO: monitor: 2023/07/14 17:11:35 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:11:35 ag(ag_cluster)[896298]: INFO: monitor: 2023/07/14 17:11:35 Replica is RESOLVING (0) >Jul 14 17:11:35 ag(ag_cluster)[896298]: INFO: monitor: 2023/07/14 17:11:35 Instance name is server3. >Jul 14 17:11:35 ag(ag_cluster)[896298]: INFO: monitor: 2023/07/14 17:11:35 Current master is server2. >Jul 14 17:11:35 ag(ag_cluster)[896298]: INFO: monitor: 2023/07/14 17:11:35 Setting the role to Secondary. >Jul 14 17:11:35 ag(ag_cluster)[896298]: INFO: monitor: 2023/07/14 17:11:35 Setting replica to SECONDARY role... >Jul 14 17:11:35 ag(ag_cluster)[896298]: INFO: monitor: 2023/07/14 17:11:35 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:11:35 ag(ag_cluster)[896298]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:11:35 ag(ag_cluster)[896298]: INFO: lease_expiry after monitor update: >Jul 14 17:11:35 ag(ag_cluster)[896298]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:11:35 ag(ag_cluster)[896298]: INFO: ag_cluster monitor : 0 >Jul 14 17:11:35 ag(ag_cluster)[896469]: INFO: mssql_validate >Jul 14 17:11:35 ag(ag_cluster)[896469]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:11:35 ag(ag_cluster)[896469]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:11:35 ag(ag_cluster)[896469]: INFO: Resource agent invoked with: monitor >Jul 14 17:11:35 ag(ag_cluster)[896469]: INFO: mssql_monitor >Jul 14 17:11:35 ag(ag_cluster)[896469]: INFO: monitor: 2023/07/14 17:11:35 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:11:35 ag(ag_cluster)[896469]: INFO: monitor: 2023/07/14 17:11:35 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:11:35 ag(ag_cluster)[896469]: INFO: monitor: 2023/07/14 17:11:35 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:11:35 ag(ag_cluster)[896469]: INFO: monitor: 2023/07/14 17:11:35 Connected to the instance at localhost:1433 >Jul 14 17:11:40 ag(ag_cluster)[896469]: INFO: monitor: 2023/07/14 17:11:40 Monitor Caller is: monitor. >Jul 14 17:11:40 ag(ag_cluster)[896469]: INFO: monitor: 2023/07/14 17:11:40 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:11:40 ag(ag_cluster)[896469]: INFO: monitor: 2023/07/14 17:11:40 Replica is RESOLVING (0) >Jul 14 17:11:40 ag(ag_cluster)[896469]: INFO: monitor: 2023/07/14 17:11:40 Instance name is server3. >Jul 14 17:11:40 ag(ag_cluster)[896469]: INFO: monitor: 2023/07/14 17:11:40 Current master is server2. >Jul 14 17:11:41 ag(ag_cluster)[896469]: INFO: monitor: 2023/07/14 17:11:40 Setting the role to Secondary. >Jul 14 17:11:41 ag(ag_cluster)[896469]: INFO: monitor: 2023/07/14 17:11:40 Setting replica to SECONDARY role... >Jul 14 17:11:41 ag(ag_cluster)[896469]: INFO: monitor: 2023/07/14 17:11:40 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:11:41 ag(ag_cluster)[896469]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:11:41 ag(ag_cluster)[896469]: INFO: lease_expiry after monitor update: >Jul 14 17:11:41 ag(ag_cluster)[896469]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:11:41 ag(ag_cluster)[896469]: INFO: ag_cluster monitor : 0 >Jul 14 17:11:42 ag(ag_cluster)[896642]: INFO: mssql_validate >Jul 14 17:11:42 ag(ag_cluster)[896642]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:11:42 ag(ag_cluster)[896642]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:11:42 ag(ag_cluster)[896642]: INFO: Resource agent invoked with: notify >Jul 14 17:11:42 ag(ag_cluster)[896642]: INFO: mssql_notify pre-start >Jul 14 17:11:43 ag(ag_cluster)[896642]: INFO: notify: 2023/07/14 17:11:42 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:11:43 ag(ag_cluster)[896642]: INFO: notify: 2023/07/14 17:11:42 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:11:43 ag(ag_cluster)[896642]: INFO: notify: 2023/07/14 17:11:43 Replica is RESOLVING (0) >Jul 14 17:11:43 ag(ag_cluster)[896642]: INFO: notify: 2023/07/14 17:11:43 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:11:43 ag(ag_cluster)[896642]: INFO: notify: 2023/07/14 17:11:43 Sequence number is 17F:F (1644972474383) >Jul 14 17:11:43 ag(ag_cluster)[896642]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:11:48 ag(ag_cluster)[896642]: INFO: ag_cluster notify : 0 >Jul 14 17:11:48 ag(ag_cluster)[896741]: INFO: mssql_validate >Jul 14 17:11:48 ag(ag_cluster)[896741]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:11:48 ag(ag_cluster)[896741]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:11:48 ag(ag_cluster)[896741]: INFO: Resource agent invoked with: monitor >Jul 14 17:11:48 ag(ag_cluster)[896741]: INFO: mssql_monitor >Jul 14 17:11:48 ag(ag_cluster)[896741]: INFO: monitor: 2023/07/14 17:11:48 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:11:48 ag(ag_cluster)[896741]: INFO: monitor: 2023/07/14 17:11:48 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:11:48 ag(ag_cluster)[896741]: INFO: monitor: 2023/07/14 17:11:48 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:11:48 ag(ag_cluster)[896741]: INFO: monitor: 2023/07/14 17:11:48 Connected to the instance at localhost:1433 >Jul 14 17:11:53 ag(ag_cluster)[896741]: INFO: monitor: 2023/07/14 17:11:53 Monitor Caller is: monitor. >Jul 14 17:11:53 ag(ag_cluster)[896741]: INFO: monitor: 2023/07/14 17:11:53 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:11:53 ag(ag_cluster)[896741]: INFO: monitor: 2023/07/14 17:11:53 Replica is RESOLVING (0) >Jul 14 17:11:53 ag(ag_cluster)[896741]: INFO: monitor: 2023/07/14 17:11:53 Instance name is server3. >Jul 14 17:11:53 ag(ag_cluster)[896741]: INFO: monitor: 2023/07/14 17:11:53 Current master is server2. >Jul 14 17:11:53 ag(ag_cluster)[896741]: INFO: monitor: 2023/07/14 17:11:53 Setting the role to Secondary. >Jul 14 17:11:53 ag(ag_cluster)[896741]: INFO: monitor: 2023/07/14 17:11:53 Setting replica to SECONDARY role... >Jul 14 17:11:53 ag(ag_cluster)[896741]: INFO: monitor: 2023/07/14 17:11:53 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:11:53 ag(ag_cluster)[896741]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:11:53 ag(ag_cluster)[896741]: INFO: lease_expiry after monitor update: >Jul 14 17:11:53 ag(ag_cluster)[896741]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:11:53 ag(ag_cluster)[896741]: INFO: ag_cluster monitor : 0 >Jul 14 17:11:53 ag(ag_cluster)[896913]: INFO: mssql_validate >Jul 14 17:11:53 ag(ag_cluster)[896913]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:11:53 ag(ag_cluster)[896913]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:11:53 ag(ag_cluster)[896913]: INFO: Resource agent invoked with: notify >Jul 14 17:11:53 ag(ag_cluster)[896913]: INFO: mssql_notify pre-start >Jul 14 17:11:53 ag(ag_cluster)[896913]: INFO: notify: 2023/07/14 17:11:53 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:11:53 ag(ag_cluster)[896913]: INFO: notify: 2023/07/14 17:11:53 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:11:53 ag(ag_cluster)[896913]: INFO: notify: 2023/07/14 17:11:53 Replica is RESOLVING (0) >Jul 14 17:11:53 ag(ag_cluster)[896913]: INFO: notify: 2023/07/14 17:11:53 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:11:53 ag(ag_cluster)[896913]: INFO: notify: 2023/07/14 17:11:53 Sequence number is 17F:F (1644972474383) >Jul 14 17:11:53 ag(ag_cluster)[896913]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:11:58 ag(ag_cluster)[896913]: INFO: ag_cluster notify : 0 >Jul 14 17:11:58 ag(ag_cluster)[897011]: INFO: mssql_validate >Jul 14 17:11:58 ag(ag_cluster)[897011]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:11:58 ag(ag_cluster)[897011]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:11:58 ag(ag_cluster)[897011]: INFO: Resource agent invoked with: monitor >Jul 14 17:11:58 ag(ag_cluster)[897011]: INFO: mssql_monitor >Jul 14 17:11:58 ag(ag_cluster)[897011]: INFO: monitor: 2023/07/14 17:11:58 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:11:58 ag(ag_cluster)[897011]: INFO: monitor: 2023/07/14 17:11:58 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:11:58 ag(ag_cluster)[897011]: INFO: monitor: 2023/07/14 17:11:58 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:11:58 ag(ag_cluster)[897011]: INFO: monitor: 2023/07/14 17:11:58 Connected to the instance at localhost:1433 >Jul 14 17:12:03 ag(ag_cluster)[897011]: INFO: monitor: 2023/07/14 17:12:03 Monitor Caller is: monitor. >Jul 14 17:12:03 ag(ag_cluster)[897011]: INFO: monitor: 2023/07/14 17:12:03 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:12:03 ag(ag_cluster)[897011]: INFO: monitor: 2023/07/14 17:12:03 Replica is RESOLVING (0) >Jul 14 17:12:03 ag(ag_cluster)[897011]: INFO: monitor: 2023/07/14 17:12:03 Instance name is server3. >Jul 14 17:12:03 ag(ag_cluster)[897011]: INFO: monitor: 2023/07/14 17:12:03 Current master is server2. >Jul 14 17:12:03 ag(ag_cluster)[897011]: INFO: monitor: 2023/07/14 17:12:03 Setting the role to Secondary. >Jul 14 17:12:03 ag(ag_cluster)[897011]: INFO: monitor: 2023/07/14 17:12:03 Setting replica to SECONDARY role... >Jul 14 17:12:03 ag(ag_cluster)[897011]: INFO: monitor: 2023/07/14 17:12:03 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:12:03 ag(ag_cluster)[897011]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:12:04 ag(ag_cluster)[897011]: INFO: lease_expiry after monitor update: >Jul 14 17:12:04 ag(ag_cluster)[897011]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:12:04 ag(ag_cluster)[897011]: INFO: ag_cluster monitor : 0 >Jul 14 17:12:04 ag(ag_cluster)[897184]: INFO: mssql_validate >Jul 14 17:12:04 ag(ag_cluster)[897184]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:12:04 ag(ag_cluster)[897184]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:12:04 ag(ag_cluster)[897184]: INFO: Resource agent invoked with: notify >Jul 14 17:12:04 ag(ag_cluster)[897184]: INFO: mssql_notify post-start >Jul 14 17:12:04 ag(ag_cluster)[897184]: INFO: ag_cluster notify : 0 >Jul 14 17:12:04 ag(ag_cluster)[897230]: INFO: mssql_validate >Jul 14 17:12:04 ag(ag_cluster)[897230]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:12:04 ag(ag_cluster)[897230]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:12:04 ag(ag_cluster)[897230]: INFO: Resource agent invoked with: notify >Jul 14 17:12:04 ag(ag_cluster)[897230]: INFO: mssql_notify pre-stop >Jul 14 17:12:04 ag(ag_cluster)[897230]: INFO: ag_cluster notify : 0 >Jul 14 17:12:04 ag(ag_cluster)[897275]: INFO: mssql_validate >Jul 14 17:12:04 ag(ag_cluster)[897275]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:12:04 ag(ag_cluster)[897275]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:12:04 ag(ag_cluster)[897275]: INFO: Resource agent invoked with: notify >Jul 14 17:12:04 ag(ag_cluster)[897275]: INFO: mssql_notify post-stop >Jul 14 17:12:04 ag(ag_cluster)[897275]: INFO: notify: 2023/07/14 17:12:04 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-stop]; connection-timeout [30]; health-threshold [3]; action [post-stop] >Jul 14 17:12:04 ag(ag_cluster)[897275]: INFO: notify: 2023/07/14 17:12:04 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:12:04 ag(ag_cluster)[897275]: INFO: notify: 2023/07/14 17:12:04 Replica is RESOLVING (0) >Jul 14 17:12:04 ag(ag_cluster)[897275]: INFO: ag_cluster notify : 0 >Jul 14 17:12:05 ag(ag_cluster)[897350]: INFO: mssql_validate >Jul 14 17:12:05 ag(ag_cluster)[897350]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:12:05 ag(ag_cluster)[897350]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:12:05 ag(ag_cluster)[897350]: INFO: Resource agent invoked with: monitor >Jul 14 17:12:05 ag(ag_cluster)[897350]: INFO: mssql_monitor >Jul 14 17:12:05 ag(ag_cluster)[897350]: INFO: monitor: 2023/07/14 17:12:05 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:12:05 ag(ag_cluster)[897350]: INFO: monitor: 2023/07/14 17:12:05 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:12:05 ag(ag_cluster)[897350]: INFO: monitor: 2023/07/14 17:12:05 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:12:05 ag(ag_cluster)[897350]: INFO: monitor: 2023/07/14 17:12:05 Connected to the instance at localhost:1433 >Jul 14 17:12:10 ag(ag_cluster)[897350]: INFO: monitor: 2023/07/14 17:12:10 Monitor Caller is: monitor. >Jul 14 17:12:10 ag(ag_cluster)[897350]: INFO: monitor: 2023/07/14 17:12:10 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:12:10 ag(ag_cluster)[897350]: INFO: monitor: 2023/07/14 17:12:10 Replica is RESOLVING (0) >Jul 14 17:12:10 ag(ag_cluster)[897350]: INFO: monitor: 2023/07/14 17:12:10 Instance name is server3. >Jul 14 17:12:10 ag(ag_cluster)[897350]: INFO: monitor: 2023/07/14 17:12:10 Current master is server2. >Jul 14 17:12:10 ag(ag_cluster)[897350]: INFO: monitor: 2023/07/14 17:12:10 Setting the role to Secondary. >Jul 14 17:12:10 ag(ag_cluster)[897350]: INFO: monitor: 2023/07/14 17:12:10 Setting replica to SECONDARY role... >Jul 14 17:12:10 ag(ag_cluster)[897350]: INFO: monitor: 2023/07/14 17:12:10 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:12:10 ag(ag_cluster)[897350]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:12:10 ag(ag_cluster)[897350]: INFO: lease_expiry after monitor update: >Jul 14 17:12:10 ag(ag_cluster)[897350]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:12:10 ag(ag_cluster)[897350]: INFO: ag_cluster monitor : 0 >Jul 14 17:12:14 ag(ag_cluster)[897522]: INFO: mssql_validate >Jul 14 17:12:14 ag(ag_cluster)[897522]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:12:14 ag(ag_cluster)[897522]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:12:14 ag(ag_cluster)[897522]: INFO: Resource agent invoked with: monitor >Jul 14 17:12:14 ag(ag_cluster)[897522]: INFO: mssql_monitor >Jul 14 17:12:14 ag(ag_cluster)[897522]: INFO: monitor: 2023/07/14 17:12:14 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:12:14 ag(ag_cluster)[897522]: INFO: monitor: 2023/07/14 17:12:14 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:12:14 ag(ag_cluster)[897522]: INFO: monitor: 2023/07/14 17:12:14 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:12:14 ag(ag_cluster)[897522]: INFO: monitor: 2023/07/14 17:12:14 Connected to the instance at localhost:1433 >Jul 14 17:12:19 ag(ag_cluster)[897522]: INFO: monitor: 2023/07/14 17:12:19 Monitor Caller is: monitor. >Jul 14 17:12:19 ag(ag_cluster)[897522]: INFO: monitor: 2023/07/14 17:12:19 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:12:19 ag(ag_cluster)[897522]: INFO: monitor: 2023/07/14 17:12:19 Replica is RESOLVING (0) >Jul 14 17:12:19 ag(ag_cluster)[897522]: INFO: monitor: 2023/07/14 17:12:19 Instance name is server3. >Jul 14 17:12:19 ag(ag_cluster)[897522]: INFO: monitor: 2023/07/14 17:12:19 Current master is server2. >Jul 14 17:12:19 ag(ag_cluster)[897522]: INFO: monitor: 2023/07/14 17:12:19 Setting the role to Secondary. >Jul 14 17:12:19 ag(ag_cluster)[897522]: INFO: monitor: 2023/07/14 17:12:19 Setting replica to SECONDARY role... >Jul 14 17:12:19 ag(ag_cluster)[897522]: INFO: monitor: 2023/07/14 17:12:19 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:12:19 ag(ag_cluster)[897522]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:12:19 ag(ag_cluster)[897522]: INFO: lease_expiry after monitor update: >Jul 14 17:12:19 ag(ag_cluster)[897522]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:12:19 ag(ag_cluster)[897522]: INFO: ag_cluster monitor : 0 >Jul 14 17:12:22 ag(ag_cluster)[897703]: INFO: mssql_validate >Jul 14 17:12:22 ag(ag_cluster)[897703]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:12:22 ag(ag_cluster)[897703]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:12:22 ag(ag_cluster)[897703]: INFO: Resource agent invoked with: monitor >Jul 14 17:12:23 ag(ag_cluster)[897703]: INFO: mssql_monitor >Jul 14 17:12:23 ag(ag_cluster)[897703]: INFO: monitor: 2023/07/14 17:12:23 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:12:23 ag(ag_cluster)[897703]: INFO: monitor: 2023/07/14 17:12:23 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:12:23 ag(ag_cluster)[897703]: INFO: monitor: 2023/07/14 17:12:23 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:12:23 ag(ag_cluster)[897703]: INFO: monitor: 2023/07/14 17:12:23 Connected to the instance at localhost:1433 >Jul 14 17:12:28 ag(ag_cluster)[897703]: INFO: monitor: 2023/07/14 17:12:28 Monitor Caller is: monitor. >Jul 14 17:12:28 ag(ag_cluster)[897703]: INFO: monitor: 2023/07/14 17:12:28 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:12:28 ag(ag_cluster)[897703]: INFO: monitor: 2023/07/14 17:12:28 Replica is RESOLVING (0) >Jul 14 17:12:28 ag(ag_cluster)[897703]: INFO: monitor: 2023/07/14 17:12:28 Instance name is server3. >Jul 14 17:12:28 ag(ag_cluster)[897703]: INFO: monitor: 2023/07/14 17:12:28 Current master is server2. >Jul 14 17:12:28 ag(ag_cluster)[897703]: INFO: monitor: 2023/07/14 17:12:28 Setting the role to Secondary. >Jul 14 17:12:28 ag(ag_cluster)[897703]: INFO: monitor: 2023/07/14 17:12:28 Setting replica to SECONDARY role... >Jul 14 17:12:28 ag(ag_cluster)[897703]: INFO: monitor: 2023/07/14 17:12:28 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:12:28 ag(ag_cluster)[897703]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:12:28 ag(ag_cluster)[897703]: INFO: lease_expiry after monitor update: >Jul 14 17:12:28 ag(ag_cluster)[897703]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:12:28 ag(ag_cluster)[897703]: INFO: ag_cluster monitor : 0 >Jul 14 17:12:29 ag(ag_cluster)[897884]: INFO: mssql_validate >Jul 14 17:12:29 ag(ag_cluster)[897884]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:12:29 ag(ag_cluster)[897884]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:12:29 ag(ag_cluster)[897884]: INFO: Resource agent invoked with: monitor >Jul 14 17:12:29 ag(ag_cluster)[897884]: INFO: mssql_monitor >Jul 14 17:12:29 ag(ag_cluster)[897884]: INFO: monitor: 2023/07/14 17:12:29 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:12:29 ag(ag_cluster)[897884]: INFO: monitor: 2023/07/14 17:12:29 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:12:29 ag(ag_cluster)[897884]: INFO: monitor: 2023/07/14 17:12:29 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:12:29 ag(ag_cluster)[897884]: INFO: monitor: 2023/07/14 17:12:29 Connected to the instance at localhost:1433 >Jul 14 17:12:34 ag(ag_cluster)[897884]: INFO: monitor: 2023/07/14 17:12:34 Monitor Caller is: monitor. >Jul 14 17:12:34 ag(ag_cluster)[897884]: INFO: monitor: 2023/07/14 17:12:34 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:12:34 ag(ag_cluster)[897884]: INFO: monitor: 2023/07/14 17:12:34 Replica is RESOLVING (0) >Jul 14 17:12:34 ag(ag_cluster)[897884]: INFO: monitor: 2023/07/14 17:12:34 Instance name is server3. >Jul 14 17:12:34 ag(ag_cluster)[897884]: INFO: monitor: 2023/07/14 17:12:34 Current master is server2. >Jul 14 17:12:34 ag(ag_cluster)[897884]: INFO: monitor: 2023/07/14 17:12:34 Setting the role to Secondary. >Jul 14 17:12:34 ag(ag_cluster)[897884]: INFO: monitor: 2023/07/14 17:12:34 Setting replica to SECONDARY role... >Jul 14 17:12:34 ag(ag_cluster)[897884]: INFO: monitor: 2023/07/14 17:12:34 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:12:35 ag(ag_cluster)[897884]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:12:35 ag(ag_cluster)[897884]: INFO: lease_expiry after monitor update: >Jul 14 17:12:35 ag(ag_cluster)[897884]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:12:35 ag(ag_cluster)[897884]: INFO: ag_cluster monitor : 0 >Jul 14 17:12:40 ag(ag_cluster)[898057]: INFO: mssql_validate >Jul 14 17:12:40 ag(ag_cluster)[898057]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:12:40 ag(ag_cluster)[898057]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:12:40 ag(ag_cluster)[898057]: INFO: Resource agent invoked with: monitor >Jul 14 17:12:40 ag(ag_cluster)[898057]: INFO: mssql_monitor >Jul 14 17:12:40 ag(ag_cluster)[898057]: INFO: monitor: 2023/07/14 17:12:40 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:12:40 ag(ag_cluster)[898057]: INFO: monitor: 2023/07/14 17:12:40 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:12:40 ag(ag_cluster)[898057]: INFO: monitor: 2023/07/14 17:12:40 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:12:40 ag(ag_cluster)[898057]: INFO: monitor: 2023/07/14 17:12:40 Connected to the instance at localhost:1433 >Jul 14 17:12:45 ag(ag_cluster)[898057]: INFO: monitor: 2023/07/14 17:12:45 Monitor Caller is: monitor. >Jul 14 17:12:45 ag(ag_cluster)[898057]: INFO: monitor: 2023/07/14 17:12:45 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:12:45 ag(ag_cluster)[898057]: INFO: monitor: 2023/07/14 17:12:45 Replica is RESOLVING (0) >Jul 14 17:12:45 ag(ag_cluster)[898057]: INFO: monitor: 2023/07/14 17:12:45 Instance name is server3. >Jul 14 17:12:45 ag(ag_cluster)[898057]: INFO: monitor: 2023/07/14 17:12:45 Current master is server2. >Jul 14 17:12:45 ag(ag_cluster)[898057]: INFO: monitor: 2023/07/14 17:12:45 Setting the role to Secondary. >Jul 14 17:12:45 ag(ag_cluster)[898057]: INFO: monitor: 2023/07/14 17:12:45 Setting replica to SECONDARY role... >Jul 14 17:12:45 ag(ag_cluster)[898057]: INFO: monitor: 2023/07/14 17:12:45 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:12:45 ag(ag_cluster)[898057]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:12:45 ag(ag_cluster)[898057]: INFO: lease_expiry after monitor update: >Jul 14 17:12:45 ag(ag_cluster)[898057]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:12:45 ag(ag_cluster)[898057]: INFO: ag_cluster monitor : 0 >Jul 14 17:12:45 ag(ag_cluster)[898228]: INFO: mssql_validate >Jul 14 17:12:45 ag(ag_cluster)[898228]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:12:45 ag(ag_cluster)[898228]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:12:45 ag(ag_cluster)[898228]: INFO: Resource agent invoked with: notify >Jul 14 17:12:45 ag(ag_cluster)[898228]: INFO: mssql_notify pre-demote >Jul 14 17:12:45 ag(ag_cluster)[898228]: INFO: ag_cluster notify : 0 >Jul 14 17:12:45 ag(ag_cluster)[898273]: INFO: mssql_validate >Jul 14 17:12:45 ag(ag_cluster)[898273]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:12:46 ag(ag_cluster)[898273]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:12:46 ag(ag_cluster)[898273]: INFO: Resource agent invoked with: monitor >Jul 14 17:12:46 ag(ag_cluster)[898273]: INFO: mssql_monitor >Jul 14 17:12:46 ag(ag_cluster)[898273]: INFO: monitor: 2023/07/14 17:12:46 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:12:46 ag(ag_cluster)[898273]: INFO: monitor: 2023/07/14 17:12:46 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master []; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:12:46 ag(ag_cluster)[898273]: INFO: monitor: 2023/07/14 17:12:46 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:12:46 ag(ag_cluster)[898273]: INFO: monitor: 2023/07/14 17:12:46 Connected to the instance at localhost:1433 >Jul 14 17:12:51 ag(ag_cluster)[898273]: INFO: monitor: 2023/07/14 17:12:51 Monitor Caller is: monitor. >Jul 14 17:12:51 ag(ag_cluster)[898273]: INFO: monitor: 2023/07/14 17:12:51 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:12:51 ag(ag_cluster)[898273]: INFO: monitor: 2023/07/14 17:12:51 Replica is RESOLVING (0) >Jul 14 17:12:51 ag(ag_cluster)[898273]: INFO: monitor: 2023/07/14 17:12:51 Instance name is server3. >Jul 14 17:12:51 ag(ag_cluster)[898273]: INFO: monitor: 2023/07/14 17:12:51 Current master is . >Jul 14 17:12:51 ag(ag_cluster)[898273]: INFO: monitor: PROMOTION_SCORE: 10 >Jul 14 17:12:51 ag(ag_cluster)[898273]: INFO: lease_expiry after monitor update: >Jul 14 17:12:51 ag(ag_cluster)[898273]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:12:51 ag(ag_cluster)[898273]: INFO: ag_cluster monitor : 0 >Jul 14 17:12:51 ag(ag_cluster)[898427]: INFO: mssql_validate >Jul 14 17:12:51 ag(ag_cluster)[898427]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:12:51 ag(ag_cluster)[898427]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:12:51 ag(ag_cluster)[898427]: INFO: Resource agent invoked with: notify >Jul 14 17:12:51 ag(ag_cluster)[898427]: INFO: mssql_notify post-demote >Jul 14 17:12:51 ag(ag_cluster)[898427]: INFO: ag_cluster notify : 0 >Jul 14 17:12:51 ag(ag_cluster)[898472]: INFO: mssql_validate >Jul 14 17:12:51 ag(ag_cluster)[898472]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:12:51 ag(ag_cluster)[898472]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:12:51 ag(ag_cluster)[898472]: INFO: Resource agent invoked with: notify >Jul 14 17:12:51 ag(ag_cluster)[898472]: INFO: mssql_notify pre-stop >Jul 14 17:12:51 ag(ag_cluster)[898472]: INFO: ag_cluster notify : 0 >Jul 14 17:12:51 ag(ag_cluster)[898517]: INFO: mssql_validate >Jul 14 17:12:51 ag(ag_cluster)[898517]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:12:51 ag(ag_cluster)[898517]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:12:51 ag(ag_cluster)[898517]: INFO: Resource agent invoked with: notify >Jul 14 17:12:51 ag(ag_cluster)[898517]: INFO: mssql_notify post-stop >Jul 14 17:12:52 ag(ag_cluster)[898517]: INFO: notify: 2023/07/14 17:12:51 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-stop]; connection-timeout [30]; health-threshold [3]; action [post-stop] >Jul 14 17:12:52 ag(ag_cluster)[898517]: INFO: notify: 2023/07/14 17:12:52 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:12:52 ag(ag_cluster)[898517]: INFO: notify: 2023/07/14 17:12:52 Replica is RESOLVING (0) >Jul 14 17:12:52 ag(ag_cluster)[898517]: INFO: ag_cluster notify : 0 >Jul 14 17:12:52 ag(ag_cluster)[898591]: INFO: mssql_validate >Jul 14 17:12:52 ag(ag_cluster)[898591]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:12:52 ag(ag_cluster)[898591]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:12:52 ag(ag_cluster)[898591]: INFO: Resource agent invoked with: notify >Jul 14 17:12:52 ag(ag_cluster)[898591]: INFO: mssql_notify pre-start >Jul 14 17:12:52 ag(ag_cluster)[898591]: INFO: notify: 2023/07/14 17:12:52 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:12:52 ag(ag_cluster)[898591]: INFO: notify: 2023/07/14 17:12:52 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:12:52 ag(ag_cluster)[898591]: INFO: notify: 2023/07/14 17:12:52 Replica is RESOLVING (0) >Jul 14 17:12:52 ag(ag_cluster)[898591]: INFO: notify: 2023/07/14 17:12:52 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:12:52 ag(ag_cluster)[898591]: INFO: notify: 2023/07/14 17:12:52 Sequence number is 17F:F (1644972474383) >Jul 14 17:12:52 ag(ag_cluster)[898591]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:12:57 ag(ag_cluster)[898591]: INFO: ag_cluster notify : 0 >Jul 14 17:12:57 ag(ag_cluster)[898691]: INFO: mssql_validate >Jul 14 17:12:57 ag(ag_cluster)[898691]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:12:57 ag(ag_cluster)[898691]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:12:57 ag(ag_cluster)[898691]: INFO: Resource agent invoked with: notify >Jul 14 17:12:57 ag(ag_cluster)[898691]: INFO: mssql_notify pre-start >Jul 14 17:12:57 ag(ag_cluster)[898691]: INFO: notify: 2023/07/14 17:12:57 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:12:57 ag(ag_cluster)[898691]: INFO: notify: 2023/07/14 17:12:57 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:12:57 ag(ag_cluster)[898691]: INFO: notify: 2023/07/14 17:12:57 Replica is RESOLVING (0) >Jul 14 17:12:57 ag(ag_cluster)[898691]: INFO: notify: 2023/07/14 17:12:57 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:12:57 ag(ag_cluster)[898691]: INFO: notify: 2023/07/14 17:12:57 Sequence number is 17F:F (1644972474383) >Jul 14 17:12:57 ag(ag_cluster)[898691]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:13:02 ag(ag_cluster)[898691]: INFO: ag_cluster notify : 0 >Jul 14 17:13:08 ag(ag_cluster)[898792]: INFO: mssql_validate >Jul 14 17:13:08 ag(ag_cluster)[898792]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:13:08 ag(ag_cluster)[898792]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:13:08 ag(ag_cluster)[898792]: INFO: Resource agent invoked with: notify >Jul 14 17:13:08 ag(ag_cluster)[898792]: INFO: mssql_notify post-start >Jul 14 17:13:08 ag(ag_cluster)[898792]: INFO: ag_cluster notify : 0 >Jul 14 17:13:08 ag(ag_cluster)[898838]: INFO: mssql_validate >Jul 14 17:13:08 ag(ag_cluster)[898838]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:13:08 ag(ag_cluster)[898838]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:13:08 ag(ag_cluster)[898838]: INFO: Resource agent invoked with: notify >Jul 14 17:13:08 ag(ag_cluster)[898838]: INFO: mssql_notify pre-start >Jul 14 17:13:08 ag(ag_cluster)[898838]: INFO: notify: 2023/07/14 17:13:08 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:13:08 ag(ag_cluster)[898838]: INFO: notify: 2023/07/14 17:13:08 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:13:08 ag(ag_cluster)[898838]: INFO: notify: 2023/07/14 17:13:08 Replica is RESOLVING (0) >Jul 14 17:13:08 ag(ag_cluster)[898838]: INFO: notify: 2023/07/14 17:13:08 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:13:08 ag(ag_cluster)[898838]: INFO: notify: 2023/07/14 17:13:08 Sequence number is 17F:F (1644972474383) >Jul 14 17:13:08 ag(ag_cluster)[898838]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:13:13 ag(ag_cluster)[898838]: INFO: ag_cluster notify : 0 >Jul 14 17:13:13 ag(ag_cluster)[898937]: INFO: mssql_validate >Jul 14 17:13:13 ag(ag_cluster)[898937]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:13:13 ag(ag_cluster)[898937]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:13:13 ag(ag_cluster)[898937]: INFO: Resource agent invoked with: notify >Jul 14 17:13:13 ag(ag_cluster)[898937]: INFO: mssql_notify pre-start >Jul 14 17:13:13 ag(ag_cluster)[898937]: INFO: notify: 2023/07/14 17:13:13 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:13:13 ag(ag_cluster)[898937]: INFO: notify: 2023/07/14 17:13:13 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:13:13 ag(ag_cluster)[898937]: INFO: notify: 2023/07/14 17:13:13 Replica is RESOLVING (0) >Jul 14 17:13:13 ag(ag_cluster)[898937]: INFO: notify: 2023/07/14 17:13:13 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:13:13 ag(ag_cluster)[898937]: INFO: notify: 2023/07/14 17:13:13 Sequence number is 17F:F (1644972474383) >Jul 14 17:13:13 ag(ag_cluster)[898937]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:13:18 ag(ag_cluster)[898937]: INFO: ag_cluster notify : 0 >Jul 14 17:13:19 ag(ag_cluster)[899042]: INFO: mssql_validate >Jul 14 17:13:19 ag(ag_cluster)[899042]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:13:19 ag(ag_cluster)[899042]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:13:19 ag(ag_cluster)[899042]: INFO: Resource agent invoked with: notify >Jul 14 17:13:19 ag(ag_cluster)[899042]: INFO: mssql_notify post-start >Jul 14 17:13:19 ag(ag_cluster)[899042]: INFO: ag_cluster notify : 0 >Jul 14 17:13:19 ag(ag_cluster)[899088]: INFO: mssql_validate >Jul 14 17:13:19 ag(ag_cluster)[899088]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:13:19 ag(ag_cluster)[899088]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:13:19 ag(ag_cluster)[899088]: INFO: Resource agent invoked with: notify >Jul 14 17:13:19 ag(ag_cluster)[899088]: INFO: mssql_notify pre-stop >Jul 14 17:13:19 ag(ag_cluster)[899088]: INFO: ag_cluster notify : 0 >Jul 14 17:13:19 ag(ag_cluster)[899133]: INFO: mssql_validate >Jul 14 17:13:19 ag(ag_cluster)[899133]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:13:19 ag(ag_cluster)[899133]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:13:19 ag(ag_cluster)[899133]: INFO: Resource agent invoked with: notify >Jul 14 17:13:19 ag(ag_cluster)[899133]: INFO: mssql_notify post-stop >Jul 14 17:13:19 ag(ag_cluster)[899133]: INFO: notify: 2023/07/14 17:13:19 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-stop]; connection-timeout [30]; health-threshold [3]; action [post-stop] >Jul 14 17:13:19 ag(ag_cluster)[899133]: INFO: notify: 2023/07/14 17:13:19 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:13:19 ag(ag_cluster)[899133]: INFO: notify: 2023/07/14 17:13:19 Replica is RESOLVING (0) >Jul 14 17:13:19 ag(ag_cluster)[899133]: INFO: ag_cluster notify : 0 >Jul 14 17:13:19 ag(ag_cluster)[899208]: INFO: mssql_validate >Jul 14 17:13:19 ag(ag_cluster)[899208]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:13:19 ag(ag_cluster)[899208]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:13:19 ag(ag_cluster)[899208]: INFO: Resource agent invoked with: notify >Jul 14 17:13:19 ag(ag_cluster)[899208]: INFO: mssql_notify pre-promote >Jul 14 17:13:19 ag(ag_cluster)[899208]: INFO: notify: 2023/07/14 17:13:19 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-promote]; connection-timeout [30]; health-threshold [3]; action [pre-promote] >Jul 14 17:13:19 ag(ag_cluster)[899208]: INFO: notify: 2023/07/14 17:13:19 From RetryExecute - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:13:19 ag(ag_cluster)[899208]: INFO: notify: 2023/07/14 17:13:19 Connected to the instance at localhost:1433 >Jul 14 17:13:24 ag(ag_cluster)[899208]: INFO: notify: 2023/07/14 17:13:24 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:13:24 ag(ag_cluster)[899208]: INFO: notify: 2023/07/14 17:13:24 Sequence number is 17F:F (1644972474383) >Jul 14 17:13:24 ag(ag_cluster)[899208]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:13:29 ag(ag_cluster)[899208]: INFO: ag_cluster notify : 0 >Jul 14 17:13:30 ag(ag_cluster)[899321]: INFO: mssql_validate >Jul 14 17:13:30 ag(ag_cluster)[899321]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:13:30 ag(ag_cluster)[899321]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:13:30 ag(ag_cluster)[899321]: INFO: Resource agent invoked with: notify >Jul 14 17:13:30 ag(ag_cluster)[899321]: INFO: mssql_notify pre-promote >Jul 14 17:13:30 ag(ag_cluster)[899321]: INFO: notify: 2023/07/14 17:13:30 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-promote]; connection-timeout [30]; health-threshold [3]; action [pre-promote] >Jul 14 17:13:30 ag(ag_cluster)[899321]: INFO: notify: 2023/07/14 17:13:30 From RetryExecute - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:13:30 ag(ag_cluster)[899321]: INFO: notify: 2023/07/14 17:13:30 Connected to the instance at localhost:1433 >Jul 14 17:13:35 ag(ag_cluster)[899321]: INFO: notify: 2023/07/14 17:13:35 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:13:35 ag(ag_cluster)[899321]: INFO: notify: 2023/07/14 17:13:35 Sequence number is 17F:F (1644972474383) >Jul 14 17:13:35 ag(ag_cluster)[899321]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:13:40 ag(ag_cluster)[899321]: INFO: ag_cluster notify : 0 >Jul 14 17:13:45 ag(ag_cluster)[899425]: INFO: mssql_validate >Jul 14 17:13:45 ag(ag_cluster)[899425]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:13:45 ag(ag_cluster)[899425]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:13:45 ag(ag_cluster)[899425]: INFO: Resource agent invoked with: notify >Jul 14 17:13:45 ag(ag_cluster)[899425]: INFO: mssql_notify post-promote >Jul 14 17:13:45 ag(ag_cluster)[899425]: INFO: notify: 2023/07/14 17:13:45 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-promote]; connection-timeout [30]; health-threshold [3]; action [post-promote] >Jul 14 17:13:45 ag(ag_cluster)[899425]: INFO: notify: 2023/07/14 17:13:45 [DEBUG] AG Helper PostPromote Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:13:45 ag(ag_cluster)[899425]: INFO: notify: 2023/07/14 17:13:45 Replica is RESOLVING (0) >Jul 14 17:13:45 ag(ag_cluster)[899425]: INFO: ag_cluster notify : 0 >Jul 14 17:13:46 ag(ag_cluster)[899500]: INFO: mssql_validate >Jul 14 17:13:46 ag(ag_cluster)[899500]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:13:46 ag(ag_cluster)[899500]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:13:46 ag(ag_cluster)[899500]: INFO: Resource agent invoked with: monitor >Jul 14 17:13:46 ag(ag_cluster)[899500]: INFO: mssql_monitor >Jul 14 17:13:46 ag(ag_cluster)[899500]: INFO: monitor: 2023/07/14 17:13:46 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:13:46 ag(ag_cluster)[899500]: INFO: monitor: 2023/07/14 17:13:46 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:13:46 ag(ag_cluster)[899500]: INFO: monitor: 2023/07/14 17:13:46 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:13:46 ag(ag_cluster)[899500]: INFO: monitor: 2023/07/14 17:13:46 Connected to the instance at localhost:1433 >Jul 14 17:13:51 ag(ag_cluster)[899500]: INFO: monitor: 2023/07/14 17:13:51 Monitor Caller is: monitor. >Jul 14 17:13:51 ag(ag_cluster)[899500]: INFO: monitor: 2023/07/14 17:13:51 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:13:51 ag(ag_cluster)[899500]: INFO: monitor: 2023/07/14 17:13:51 Replica is RESOLVING (0) >Jul 14 17:13:51 ag(ag_cluster)[899500]: INFO: monitor: 2023/07/14 17:13:51 Instance name is server3. >Jul 14 17:13:51 ag(ag_cluster)[899500]: INFO: monitor: 2023/07/14 17:13:51 Current master is server2. >Jul 14 17:13:51 ag(ag_cluster)[899500]: INFO: monitor: 2023/07/14 17:13:51 Setting the role to Secondary. >Jul 14 17:13:51 ag(ag_cluster)[899500]: INFO: monitor: 2023/07/14 17:13:51 Setting replica to SECONDARY role... >Jul 14 17:13:51 ag(ag_cluster)[899500]: INFO: monitor: 2023/07/14 17:13:51 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:13:51 ag(ag_cluster)[899500]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:13:51 ag(ag_cluster)[899500]: INFO: lease_expiry after monitor update: >Jul 14 17:13:51 ag(ag_cluster)[899500]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:13:51 ag(ag_cluster)[899500]: INFO: ag_cluster monitor : 0 >Jul 14 17:13:51 ag(ag_cluster)[899672]: INFO: mssql_validate >Jul 14 17:13:51 ag(ag_cluster)[899672]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:13:51 ag(ag_cluster)[899672]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:13:51 ag(ag_cluster)[899672]: INFO: Resource agent invoked with: monitor >Jul 14 17:13:51 ag(ag_cluster)[899672]: INFO: mssql_monitor >Jul 14 17:13:51 ag(ag_cluster)[899672]: INFO: monitor: 2023/07/14 17:13:51 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:13:51 ag(ag_cluster)[899672]: INFO: monitor: 2023/07/14 17:13:51 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:13:51 ag(ag_cluster)[899672]: INFO: monitor: 2023/07/14 17:13:51 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:13:51 ag(ag_cluster)[899672]: INFO: monitor: 2023/07/14 17:13:51 Connected to the instance at localhost:1433 >Jul 14 17:13:56 ag(ag_cluster)[899672]: INFO: monitor: 2023/07/14 17:13:56 Monitor Caller is: monitor. >Jul 14 17:13:56 ag(ag_cluster)[899672]: INFO: monitor: 2023/07/14 17:13:56 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:13:56 ag(ag_cluster)[899672]: INFO: monitor: 2023/07/14 17:13:56 Replica is RESOLVING (0) >Jul 14 17:13:56 ag(ag_cluster)[899672]: INFO: monitor: 2023/07/14 17:13:56 Instance name is server3. >Jul 14 17:13:56 ag(ag_cluster)[899672]: INFO: monitor: 2023/07/14 17:13:56 Current master is server2. >Jul 14 17:13:56 ag(ag_cluster)[899672]: INFO: monitor: 2023/07/14 17:13:56 Setting the role to Secondary. >Jul 14 17:13:56 ag(ag_cluster)[899672]: INFO: monitor: 2023/07/14 17:13:56 Setting replica to SECONDARY role... >Jul 14 17:13:56 ag(ag_cluster)[899672]: INFO: monitor: 2023/07/14 17:13:56 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:13:56 ag(ag_cluster)[899672]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:13:56 ag(ag_cluster)[899672]: INFO: lease_expiry after monitor update: >Jul 14 17:13:56 ag(ag_cluster)[899672]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:13:56 ag(ag_cluster)[899672]: INFO: ag_cluster monitor : 0 >Jul 14 17:14:03 ag(ag_cluster)[899849]: INFO: mssql_validate >Jul 14 17:14:03 ag(ag_cluster)[899849]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:14:03 ag(ag_cluster)[899849]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:14:03 ag(ag_cluster)[899849]: INFO: Resource agent invoked with: monitor >Jul 14 17:14:03 ag(ag_cluster)[899849]: INFO: mssql_monitor >Jul 14 17:14:03 ag(ag_cluster)[899849]: INFO: monitor: 2023/07/14 17:14:03 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:14:03 ag(ag_cluster)[899849]: INFO: monitor: 2023/07/14 17:14:03 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:14:03 ag(ag_cluster)[899849]: INFO: monitor: 2023/07/14 17:14:03 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:14:03 ag(ag_cluster)[899849]: INFO: monitor: 2023/07/14 17:14:03 Connected to the instance at localhost:1433 >Jul 14 17:14:08 ag(ag_cluster)[899849]: INFO: monitor: 2023/07/14 17:14:08 Monitor Caller is: monitor. >Jul 14 17:14:08 ag(ag_cluster)[899849]: INFO: monitor: 2023/07/14 17:14:08 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:14:08 ag(ag_cluster)[899849]: INFO: monitor: 2023/07/14 17:14:08 Replica is RESOLVING (0) >Jul 14 17:14:08 ag(ag_cluster)[899849]: INFO: monitor: 2023/07/14 17:14:08 Instance name is server3. >Jul 14 17:14:08 ag(ag_cluster)[899849]: INFO: monitor: 2023/07/14 17:14:08 Current master is server2. >Jul 14 17:14:08 ag(ag_cluster)[899849]: INFO: monitor: 2023/07/14 17:14:08 Setting the role to Secondary. >Jul 14 17:14:08 ag(ag_cluster)[899849]: INFO: monitor: 2023/07/14 17:14:08 Setting replica to SECONDARY role... >Jul 14 17:14:08 ag(ag_cluster)[899849]: INFO: monitor: 2023/07/14 17:14:08 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:14:08 ag(ag_cluster)[899849]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:14:08 ag(ag_cluster)[899849]: INFO: lease_expiry after monitor update: >Jul 14 17:14:08 ag(ag_cluster)[899849]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:14:08 ag(ag_cluster)[899849]: INFO: ag_cluster monitor : 0 >Jul 14 17:14:08 ag(ag_cluster)[900023]: INFO: mssql_validate >Jul 14 17:14:08 ag(ag_cluster)[900023]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:14:08 ag(ag_cluster)[900023]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:14:08 ag(ag_cluster)[900023]: INFO: Resource agent invoked with: monitor >Jul 14 17:14:08 ag(ag_cluster)[900023]: INFO: mssql_monitor >Jul 14 17:14:08 ag(ag_cluster)[900023]: INFO: monitor: 2023/07/14 17:14:08 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:14:09 ag(ag_cluster)[900023]: INFO: monitor: 2023/07/14 17:14:08 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:14:09 ag(ag_cluster)[900023]: INFO: monitor: 2023/07/14 17:14:08 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:14:09 ag(ag_cluster)[900023]: INFO: monitor: 2023/07/14 17:14:09 Connected to the instance at localhost:1433 >Jul 14 17:14:14 ag(ag_cluster)[900023]: INFO: monitor: 2023/07/14 17:14:14 Monitor Caller is: monitor. >Jul 14 17:14:14 ag(ag_cluster)[900023]: INFO: monitor: 2023/07/14 17:14:14 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:14:14 ag(ag_cluster)[900023]: INFO: monitor: 2023/07/14 17:14:14 Replica is RESOLVING (0) >Jul 14 17:14:14 ag(ag_cluster)[900023]: INFO: monitor: 2023/07/14 17:14:14 Instance name is server3. >Jul 14 17:14:14 ag(ag_cluster)[900023]: INFO: monitor: 2023/07/14 17:14:14 Current master is server2. >Jul 14 17:14:14 ag(ag_cluster)[900023]: INFO: monitor: 2023/07/14 17:14:14 Setting the role to Secondary. >Jul 14 17:14:14 ag(ag_cluster)[900023]: INFO: monitor: 2023/07/14 17:14:14 Setting replica to SECONDARY role... >Jul 14 17:14:14 ag(ag_cluster)[900023]: INFO: monitor: 2023/07/14 17:14:14 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:14:14 ag(ag_cluster)[900023]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:14:14 ag(ag_cluster)[900023]: INFO: lease_expiry after monitor update: >Jul 14 17:14:14 ag(ag_cluster)[900023]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:14:14 ag(ag_cluster)[900023]: INFO: ag_cluster monitor : 0 >Jul 14 17:14:20 ag(ag_cluster)[900203]: INFO: mssql_validate >Jul 14 17:14:20 ag(ag_cluster)[900203]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:14:20 ag(ag_cluster)[900203]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:14:20 ag(ag_cluster)[900203]: INFO: Resource agent invoked with: monitor >Jul 14 17:14:20 ag(ag_cluster)[900203]: INFO: mssql_monitor >Jul 14 17:14:21 ag(ag_cluster)[900203]: INFO: monitor: 2023/07/14 17:14:21 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:14:21 ag(ag_cluster)[900203]: INFO: monitor: 2023/07/14 17:14:21 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:14:21 ag(ag_cluster)[900203]: INFO: monitor: 2023/07/14 17:14:21 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:14:21 ag(ag_cluster)[900203]: INFO: monitor: 2023/07/14 17:14:21 Connected to the instance at localhost:1433 >Jul 14 17:14:26 ag(ag_cluster)[900203]: INFO: monitor: 2023/07/14 17:14:26 Monitor Caller is: monitor. >Jul 14 17:14:26 ag(ag_cluster)[900203]: INFO: monitor: 2023/07/14 17:14:26 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:14:26 ag(ag_cluster)[900203]: INFO: monitor: 2023/07/14 17:14:26 Replica is RESOLVING (0) >Jul 14 17:14:26 ag(ag_cluster)[900203]: INFO: monitor: 2023/07/14 17:14:26 Instance name is server3. >Jul 14 17:14:26 ag(ag_cluster)[900203]: INFO: monitor: 2023/07/14 17:14:26 Current master is server2. >Jul 14 17:14:26 ag(ag_cluster)[900203]: INFO: monitor: 2023/07/14 17:14:26 Setting the role to Secondary. >Jul 14 17:14:26 ag(ag_cluster)[900203]: INFO: monitor: 2023/07/14 17:14:26 Setting replica to SECONDARY role... >Jul 14 17:14:26 ag(ag_cluster)[900203]: INFO: monitor: 2023/07/14 17:14:26 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:14:26 ag(ag_cluster)[900203]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:14:26 ag(ag_cluster)[900203]: INFO: lease_expiry after monitor update: >Jul 14 17:14:26 ag(ag_cluster)[900203]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:14:26 ag(ag_cluster)[900203]: INFO: ag_cluster monitor : 0 >Jul 14 17:14:26 ag(ag_cluster)[900375]: INFO: mssql_validate >Jul 14 17:14:26 ag(ag_cluster)[900375]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:14:26 ag(ag_cluster)[900375]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:14:26 ag(ag_cluster)[900375]: INFO: Resource agent invoked with: notify >Jul 14 17:14:26 ag(ag_cluster)[900375]: INFO: mssql_notify pre-start >Jul 14 17:14:26 ag(ag_cluster)[900375]: INFO: notify: 2023/07/14 17:14:26 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:14:26 ag(ag_cluster)[900375]: INFO: notify: 2023/07/14 17:14:26 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:14:26 ag(ag_cluster)[900375]: INFO: notify: 2023/07/14 17:14:26 Replica is RESOLVING (0) >Jul 14 17:14:26 ag(ag_cluster)[900375]: INFO: notify: 2023/07/14 17:14:26 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:14:26 ag(ag_cluster)[900375]: INFO: notify: 2023/07/14 17:14:26 Sequence number is 17F:F (1644972474383) >Jul 14 17:14:26 ag(ag_cluster)[900375]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:14:31 ag(ag_cluster)[900375]: INFO: ag_cluster notify : 0 >Jul 14 17:14:31 ag(ag_cluster)[900483]: INFO: mssql_validate >Jul 14 17:14:31 ag(ag_cluster)[900483]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:14:31 ag(ag_cluster)[900483]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:14:31 ag(ag_cluster)[900483]: INFO: Resource agent invoked with: monitor >Jul 14 17:14:31 ag(ag_cluster)[900483]: INFO: mssql_monitor >Jul 14 17:14:31 ag(ag_cluster)[900483]: INFO: monitor: 2023/07/14 17:14:31 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:14:31 ag(ag_cluster)[900483]: INFO: monitor: 2023/07/14 17:14:31 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:14:31 ag(ag_cluster)[900483]: INFO: monitor: 2023/07/14 17:14:31 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:14:31 ag(ag_cluster)[900483]: INFO: monitor: 2023/07/14 17:14:31 Connected to the instance at localhost:1433 >Jul 14 17:14:36 ag(ag_cluster)[900483]: INFO: monitor: 2023/07/14 17:14:36 Monitor Caller is: monitor. >Jul 14 17:14:36 ag(ag_cluster)[900483]: INFO: monitor: 2023/07/14 17:14:36 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:14:36 ag(ag_cluster)[900483]: INFO: monitor: 2023/07/14 17:14:36 Replica is RESOLVING (0) >Jul 14 17:14:36 ag(ag_cluster)[900483]: INFO: monitor: 2023/07/14 17:14:36 Instance name is server3. >Jul 14 17:14:36 ag(ag_cluster)[900483]: INFO: monitor: 2023/07/14 17:14:36 Current master is server2. >Jul 14 17:14:36 ag(ag_cluster)[900483]: INFO: monitor: 2023/07/14 17:14:36 Setting the role to Secondary. >Jul 14 17:14:36 ag(ag_cluster)[900483]: INFO: monitor: 2023/07/14 17:14:36 Setting replica to SECONDARY role... >Jul 14 17:14:36 ag(ag_cluster)[900483]: INFO: monitor: 2023/07/14 17:14:36 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:14:36 ag(ag_cluster)[900483]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:14:36 ag(ag_cluster)[900483]: INFO: lease_expiry after monitor update: >Jul 14 17:14:36 ag(ag_cluster)[900483]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:14:36 ag(ag_cluster)[900483]: INFO: ag_cluster monitor : 0 >Jul 14 17:14:36 ag(ag_cluster)[900655]: INFO: mssql_validate >Jul 14 17:14:36 ag(ag_cluster)[900655]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:14:36 ag(ag_cluster)[900655]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:14:36 ag(ag_cluster)[900655]: INFO: Resource agent invoked with: notify >Jul 14 17:14:36 ag(ag_cluster)[900655]: INFO: mssql_notify pre-start >Jul 14 17:14:36 ag(ag_cluster)[900655]: INFO: notify: 2023/07/14 17:14:36 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:14:36 ag(ag_cluster)[900655]: INFO: notify: 2023/07/14 17:14:36 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:14:36 ag(ag_cluster)[900655]: INFO: notify: 2023/07/14 17:14:36 Replica is RESOLVING (0) >Jul 14 17:14:36 ag(ag_cluster)[900655]: INFO: notify: 2023/07/14 17:14:36 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:14:36 ag(ag_cluster)[900655]: INFO: notify: 2023/07/14 17:14:36 Sequence number is 17F:F (1644972474383) >Jul 14 17:14:36 ag(ag_cluster)[900655]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:14:42 ag(ag_cluster)[900655]: INFO: ag_cluster notify : 0 >Jul 14 17:14:42 ag(ag_cluster)[900754]: INFO: mssql_validate >Jul 14 17:14:42 ag(ag_cluster)[900754]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:14:42 ag(ag_cluster)[900754]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:14:42 ag(ag_cluster)[900754]: INFO: Resource agent invoked with: monitor >Jul 14 17:14:42 ag(ag_cluster)[900754]: INFO: mssql_monitor >Jul 14 17:14:42 ag(ag_cluster)[900754]: INFO: monitor: 2023/07/14 17:14:42 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:14:42 ag(ag_cluster)[900754]: INFO: monitor: 2023/07/14 17:14:42 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:14:42 ag(ag_cluster)[900754]: INFO: monitor: 2023/07/14 17:14:42 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:14:42 ag(ag_cluster)[900754]: INFO: monitor: 2023/07/14 17:14:42 Connected to the instance at localhost:1433 >Jul 14 17:14:47 ag(ag_cluster)[900754]: INFO: monitor: 2023/07/14 17:14:47 Monitor Caller is: monitor. >Jul 14 17:14:47 ag(ag_cluster)[900754]: INFO: monitor: 2023/07/14 17:14:47 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:14:47 ag(ag_cluster)[900754]: INFO: monitor: 2023/07/14 17:14:47 Replica is RESOLVING (0) >Jul 14 17:14:47 ag(ag_cluster)[900754]: INFO: monitor: 2023/07/14 17:14:47 Instance name is server3. >Jul 14 17:14:47 ag(ag_cluster)[900754]: INFO: monitor: 2023/07/14 17:14:47 Current master is server2. >Jul 14 17:14:47 ag(ag_cluster)[900754]: INFO: monitor: 2023/07/14 17:14:47 Setting the role to Secondary. >Jul 14 17:14:47 ag(ag_cluster)[900754]: INFO: monitor: 2023/07/14 17:14:47 Setting replica to SECONDARY role... >Jul 14 17:14:47 ag(ag_cluster)[900754]: INFO: monitor: 2023/07/14 17:14:47 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:14:47 ag(ag_cluster)[900754]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:14:47 ag(ag_cluster)[900754]: INFO: lease_expiry after monitor update: >Jul 14 17:14:47 ag(ag_cluster)[900754]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:14:47 ag(ag_cluster)[900754]: INFO: ag_cluster monitor : 0 >Jul 14 17:14:47 ag(ag_cluster)[900926]: INFO: mssql_validate >Jul 14 17:14:47 ag(ag_cluster)[900926]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:14:47 ag(ag_cluster)[900926]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:14:47 ag(ag_cluster)[900926]: INFO: Resource agent invoked with: notify >Jul 14 17:14:47 ag(ag_cluster)[900926]: INFO: mssql_notify post-start >Jul 14 17:14:47 ag(ag_cluster)[900926]: INFO: ag_cluster notify : 0 >Jul 14 17:14:47 ag(ag_cluster)[900972]: INFO: mssql_validate >Jul 14 17:14:47 ag(ag_cluster)[900972]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:14:47 ag(ag_cluster)[900972]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:14:47 ag(ag_cluster)[900972]: INFO: Resource agent invoked with: monitor >Jul 14 17:14:47 ag(ag_cluster)[900972]: INFO: mssql_monitor >Jul 14 17:14:47 ag(ag_cluster)[900972]: INFO: monitor: 2023/07/14 17:14:47 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:14:47 ag(ag_cluster)[900972]: INFO: monitor: 2023/07/14 17:14:47 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:14:47 ag(ag_cluster)[900972]: INFO: monitor: 2023/07/14 17:14:47 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:14:47 ag(ag_cluster)[900972]: INFO: monitor: 2023/07/14 17:14:47 Connected to the instance at localhost:1433 >Jul 14 17:14:52 ag(ag_cluster)[900972]: INFO: monitor: 2023/07/14 17:14:52 Monitor Caller is: monitor. >Jul 14 17:14:52 ag(ag_cluster)[900972]: INFO: monitor: 2023/07/14 17:14:52 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:14:52 ag(ag_cluster)[900972]: INFO: monitor: 2023/07/14 17:14:52 Replica is RESOLVING (0) >Jul 14 17:14:52 ag(ag_cluster)[900972]: INFO: monitor: 2023/07/14 17:14:52 Instance name is server3. >Jul 14 17:14:52 ag(ag_cluster)[900972]: INFO: monitor: 2023/07/14 17:14:52 Current master is server2. >Jul 14 17:14:52 ag(ag_cluster)[900972]: INFO: monitor: 2023/07/14 17:14:52 Setting the role to Secondary. >Jul 14 17:14:52 ag(ag_cluster)[900972]: INFO: monitor: 2023/07/14 17:14:52 Setting replica to SECONDARY role... >Jul 14 17:14:52 ag(ag_cluster)[900972]: INFO: monitor: 2023/07/14 17:14:52 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:14:52 ag(ag_cluster)[900972]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:14:52 ag(ag_cluster)[900972]: INFO: lease_expiry after monitor update: >Jul 14 17:14:52 ag(ag_cluster)[900972]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:14:52 ag(ag_cluster)[900972]: INFO: ag_cluster monitor : 0 >Jul 14 17:14:53 ag(ag_cluster)[901144]: INFO: mssql_validate >Jul 14 17:14:53 ag(ag_cluster)[901144]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:14:53 ag(ag_cluster)[901144]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:14:53 ag(ag_cluster)[901144]: INFO: Resource agent invoked with: notify >Jul 14 17:14:53 ag(ag_cluster)[901144]: INFO: mssql_notify pre-stop >Jul 14 17:14:53 ag(ag_cluster)[901144]: INFO: ag_cluster notify : 0 >Jul 14 17:14:53 ag(ag_cluster)[901190]: INFO: mssql_validate >Jul 14 17:14:53 ag(ag_cluster)[901190]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:14:53 ag(ag_cluster)[901190]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:14:53 ag(ag_cluster)[901190]: INFO: Resource agent invoked with: notify >Jul 14 17:14:53 ag(ag_cluster)[901190]: INFO: mssql_notify post-stop >Jul 14 17:14:53 ag(ag_cluster)[901190]: INFO: notify: 2023/07/14 17:14:53 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-stop]; connection-timeout [30]; health-threshold [3]; action [post-stop] >Jul 14 17:14:53 ag(ag_cluster)[901190]: INFO: notify: 2023/07/14 17:14:53 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:14:53 ag(ag_cluster)[901190]: INFO: notify: 2023/07/14 17:14:53 Replica is RESOLVING (0) >Jul 14 17:14:53 ag(ag_cluster)[901190]: INFO: ag_cluster notify : 0 >Jul 14 17:14:59 ag(ag_cluster)[901266]: INFO: mssql_validate >Jul 14 17:14:59 ag(ag_cluster)[901266]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:14:59 ag(ag_cluster)[901266]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:14:59 ag(ag_cluster)[901266]: INFO: Resource agent invoked with: monitor >Jul 14 17:14:59 ag(ag_cluster)[901266]: INFO: mssql_monitor >Jul 14 17:14:59 ag(ag_cluster)[901266]: INFO: monitor: 2023/07/14 17:14:59 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:14:59 ag(ag_cluster)[901266]: INFO: monitor: 2023/07/14 17:14:59 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:14:59 ag(ag_cluster)[901266]: INFO: monitor: 2023/07/14 17:14:59 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:14:59 ag(ag_cluster)[901266]: INFO: monitor: 2023/07/14 17:14:59 Connected to the instance at localhost:1433 >Jul 14 17:15:04 ag(ag_cluster)[901266]: INFO: monitor: 2023/07/14 17:15:04 Monitor Caller is: monitor. >Jul 14 17:15:04 ag(ag_cluster)[901266]: INFO: monitor: 2023/07/14 17:15:04 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:15:04 ag(ag_cluster)[901266]: INFO: monitor: 2023/07/14 17:15:04 Replica is RESOLVING (0) >Jul 14 17:15:04 ag(ag_cluster)[901266]: INFO: monitor: 2023/07/14 17:15:04 Instance name is server3. >Jul 14 17:15:04 ag(ag_cluster)[901266]: INFO: monitor: 2023/07/14 17:15:04 Current master is server2. >Jul 14 17:15:04 ag(ag_cluster)[901266]: INFO: monitor: 2023/07/14 17:15:04 Setting the role to Secondary. >Jul 14 17:15:04 ag(ag_cluster)[901266]: INFO: monitor: 2023/07/14 17:15:04 Setting replica to SECONDARY role... >Jul 14 17:15:04 ag(ag_cluster)[901266]: INFO: monitor: 2023/07/14 17:15:04 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:15:04 ag(ag_cluster)[901266]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:15:04 ag(ag_cluster)[901266]: INFO: lease_expiry after monitor update: >Jul 14 17:15:04 ag(ag_cluster)[901266]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:15:04 ag(ag_cluster)[901266]: INFO: ag_cluster monitor : 0 >Jul 14 17:15:04 ag(ag_cluster)[901438]: INFO: mssql_validate >Jul 14 17:15:04 ag(ag_cluster)[901438]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:15:04 ag(ag_cluster)[901438]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:15:05 ag(ag_cluster)[901438]: INFO: Resource agent invoked with: monitor >Jul 14 17:15:05 ag(ag_cluster)[901438]: INFO: mssql_monitor >Jul 14 17:15:05 ag(ag_cluster)[901438]: INFO: monitor: 2023/07/14 17:15:05 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:15:05 ag(ag_cluster)[901438]: INFO: monitor: 2023/07/14 17:15:05 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:15:05 ag(ag_cluster)[901438]: INFO: monitor: 2023/07/14 17:15:05 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:15:05 ag(ag_cluster)[901438]: INFO: monitor: 2023/07/14 17:15:05 Connected to the instance at localhost:1433 >Jul 14 17:15:10 ag(ag_cluster)[901438]: INFO: monitor: 2023/07/14 17:15:10 Monitor Caller is: monitor. >Jul 14 17:15:10 ag(ag_cluster)[901438]: INFO: monitor: 2023/07/14 17:15:10 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:15:10 ag(ag_cluster)[901438]: INFO: monitor: 2023/07/14 17:15:10 Replica is RESOLVING (0) >Jul 14 17:15:10 ag(ag_cluster)[901438]: INFO: monitor: 2023/07/14 17:15:10 Instance name is server3. >Jul 14 17:15:10 ag(ag_cluster)[901438]: INFO: monitor: 2023/07/14 17:15:10 Current master is server2. >Jul 14 17:15:10 ag(ag_cluster)[901438]: INFO: monitor: 2023/07/14 17:15:10 Setting the role to Secondary. >Jul 14 17:15:10 ag(ag_cluster)[901438]: INFO: monitor: 2023/07/14 17:15:10 Setting replica to SECONDARY role... >Jul 14 17:15:10 ag(ag_cluster)[901438]: INFO: monitor: 2023/07/14 17:15:10 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:15:10 ag(ag_cluster)[901438]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:15:10 ag(ag_cluster)[901438]: INFO: lease_expiry after monitor update: >Jul 14 17:15:10 ag(ag_cluster)[901438]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:15:10 ag(ag_cluster)[901438]: INFO: ag_cluster monitor : 0 >Jul 14 17:15:15 ag(ag_cluster)[901616]: INFO: mssql_validate >Jul 14 17:15:15 ag(ag_cluster)[901616]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:15:15 ag(ag_cluster)[901616]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:15:15 ag(ag_cluster)[901616]: INFO: Resource agent invoked with: notify >Jul 14 17:15:15 ag(ag_cluster)[901616]: INFO: mssql_notify pre-demote >Jul 14 17:15:15 ag(ag_cluster)[901616]: INFO: ag_cluster notify : 0 >Jul 14 17:15:15 ag(ag_cluster)[901661]: INFO: mssql_validate >Jul 14 17:15:15 ag(ag_cluster)[901661]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:15:15 ag(ag_cluster)[901661]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:15:15 ag(ag_cluster)[901661]: INFO: Resource agent invoked with: notify >Jul 14 17:15:15 ag(ag_cluster)[901661]: INFO: mssql_notify post-demote >Jul 14 17:15:15 ag(ag_cluster)[901661]: INFO: ag_cluster notify : 0 >Jul 14 17:15:15 ag(ag_cluster)[901706]: INFO: mssql_validate >Jul 14 17:15:15 ag(ag_cluster)[901706]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:15:15 ag(ag_cluster)[901706]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:15:15 ag(ag_cluster)[901706]: INFO: Resource agent invoked with: notify >Jul 14 17:15:15 ag(ag_cluster)[901706]: INFO: mssql_notify pre-stop >Jul 14 17:15:15 ag(ag_cluster)[901706]: INFO: ag_cluster notify : 0 >Jul 14 17:15:15 ag(ag_cluster)[901751]: INFO: mssql_validate >Jul 14 17:15:15 ag(ag_cluster)[901751]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:15:15 ag(ag_cluster)[901751]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:15:15 ag(ag_cluster)[901751]: INFO: Resource agent invoked with: notify >Jul 14 17:15:15 ag(ag_cluster)[901751]: INFO: mssql_notify post-stop >Jul 14 17:15:15 ag(ag_cluster)[901751]: INFO: notify: 2023/07/14 17:15:15 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-stop]; connection-timeout [30]; health-threshold [3]; action [post-stop] >Jul 14 17:15:15 ag(ag_cluster)[901751]: INFO: notify: 2023/07/14 17:15:15 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:15:15 ag(ag_cluster)[901751]: INFO: notify: 2023/07/14 17:15:15 Replica is RESOLVING (0) >Jul 14 17:15:16 ag(ag_cluster)[901751]: INFO: ag_cluster notify : 0 >Jul 14 17:15:16 ag(ag_cluster)[901825]: INFO: mssql_validate >Jul 14 17:15:16 ag(ag_cluster)[901825]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:15:16 ag(ag_cluster)[901825]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:15:16 ag(ag_cluster)[901825]: INFO: Resource agent invoked with: notify >Jul 14 17:15:16 ag(ag_cluster)[901825]: INFO: mssql_notify pre-start >Jul 14 17:15:16 ag(ag_cluster)[901825]: INFO: notify: 2023/07/14 17:15:16 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:15:16 ag(ag_cluster)[901825]: INFO: notify: 2023/07/14 17:15:16 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:15:16 ag(ag_cluster)[901825]: INFO: notify: 2023/07/14 17:15:16 Replica is RESOLVING (0) >Jul 14 17:15:16 ag(ag_cluster)[901825]: INFO: notify: 2023/07/14 17:15:16 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:15:16 ag(ag_cluster)[901825]: INFO: notify: 2023/07/14 17:15:16 Sequence number is 17F:F (1644972474383) >Jul 14 17:15:16 ag(ag_cluster)[901825]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:15:21 ag(ag_cluster)[901825]: INFO: ag_cluster notify : 0 >Jul 14 17:15:21 ag(ag_cluster)[901924]: INFO: mssql_validate >Jul 14 17:15:21 ag(ag_cluster)[901924]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:15:21 ag(ag_cluster)[901924]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:15:21 ag(ag_cluster)[901924]: INFO: Resource agent invoked with: monitor >Jul 14 17:15:21 ag(ag_cluster)[901924]: INFO: mssql_monitor >Jul 14 17:15:21 ag(ag_cluster)[901924]: INFO: monitor: 2023/07/14 17:15:21 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:15:21 ag(ag_cluster)[901924]: INFO: monitor: 2023/07/14 17:15:21 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master []; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:15:21 ag(ag_cluster)[901924]: INFO: monitor: 2023/07/14 17:15:21 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:15:21 ag(ag_cluster)[901924]: INFO: monitor: 2023/07/14 17:15:21 Connected to the instance at localhost:1433 >Jul 14 17:15:26 ag(ag_cluster)[901924]: INFO: monitor: 2023/07/14 17:15:26 Monitor Caller is: monitor. >Jul 14 17:15:26 ag(ag_cluster)[901924]: INFO: monitor: 2023/07/14 17:15:26 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:15:26 ag(ag_cluster)[901924]: INFO: monitor: 2023/07/14 17:15:26 Replica is RESOLVING (0) >Jul 14 17:15:26 ag(ag_cluster)[901924]: INFO: monitor: 2023/07/14 17:15:26 Instance name is server3. >Jul 14 17:15:26 ag(ag_cluster)[901924]: INFO: monitor: 2023/07/14 17:15:26 Current master is . >Jul 14 17:15:26 ag(ag_cluster)[901924]: INFO: monitor: PROMOTION_SCORE: 10 >Jul 14 17:15:26 ag(ag_cluster)[901924]: INFO: lease_expiry after monitor update: >Jul 14 17:15:26 ag(ag_cluster)[901924]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:15:26 ag(ag_cluster)[901924]: INFO: ag_cluster monitor : 0 >Jul 14 17:15:26 ag(ag_cluster)[902078]: INFO: mssql_validate >Jul 14 17:15:26 ag(ag_cluster)[902078]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:15:26 ag(ag_cluster)[902078]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:15:26 ag(ag_cluster)[902078]: INFO: Resource agent invoked with: monitor >Jul 14 17:15:26 ag(ag_cluster)[902078]: INFO: mssql_monitor >Jul 14 17:15:26 ag(ag_cluster)[902078]: INFO: monitor: 2023/07/14 17:15:26 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:15:26 ag(ag_cluster)[902078]: INFO: monitor: 2023/07/14 17:15:26 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master []; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:15:26 ag(ag_cluster)[902078]: INFO: monitor: 2023/07/14 17:15:26 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:15:27 ag(ag_cluster)[902078]: INFO: monitor: 2023/07/14 17:15:26 Connected to the instance at localhost:1433 >Jul 14 17:15:32 ag(ag_cluster)[902078]: INFO: monitor: 2023/07/14 17:15:32 Monitor Caller is: monitor. >Jul 14 17:15:32 ag(ag_cluster)[902078]: INFO: monitor: 2023/07/14 17:15:32 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:15:32 ag(ag_cluster)[902078]: INFO: monitor: 2023/07/14 17:15:32 Replica is RESOLVING (0) >Jul 14 17:15:32 ag(ag_cluster)[902078]: INFO: monitor: 2023/07/14 17:15:32 Instance name is server3. >Jul 14 17:15:32 ag(ag_cluster)[902078]: INFO: monitor: 2023/07/14 17:15:32 Current master is . >Jul 14 17:15:32 ag(ag_cluster)[902078]: INFO: monitor: PROMOTION_SCORE: 10 >Jul 14 17:15:32 ag(ag_cluster)[902078]: INFO: lease_expiry after monitor update: >Jul 14 17:15:32 ag(ag_cluster)[902078]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:15:32 ag(ag_cluster)[902078]: INFO: ag_cluster monitor : 0 >Jul 14 17:15:32 ag(ag_cluster)[902240]: INFO: mssql_validate >Jul 14 17:15:32 ag(ag_cluster)[902240]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:15:32 ag(ag_cluster)[902240]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:15:32 ag(ag_cluster)[902240]: INFO: Resource agent invoked with: notify >Jul 14 17:15:32 ag(ag_cluster)[902240]: INFO: mssql_notify pre-start >Jul 14 17:15:32 ag(ag_cluster)[902240]: INFO: notify: 2023/07/14 17:15:32 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:15:32 ag(ag_cluster)[902240]: INFO: notify: 2023/07/14 17:15:32 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:15:32 ag(ag_cluster)[902240]: INFO: notify: 2023/07/14 17:15:32 Replica is RESOLVING (0) >Jul 14 17:15:32 ag(ag_cluster)[902240]: INFO: notify: 2023/07/14 17:15:32 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:15:32 ag(ag_cluster)[902240]: INFO: notify: 2023/07/14 17:15:32 Sequence number is 17F:F (1644972474383) >Jul 14 17:15:32 ag(ag_cluster)[902240]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:15:37 ag(ag_cluster)[902240]: INFO: ag_cluster notify : 0 >Jul 14 17:15:37 ag(ag_cluster)[902340]: INFO: mssql_validate >Jul 14 17:15:37 ag(ag_cluster)[902340]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:15:37 ag(ag_cluster)[902340]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:15:37 ag(ag_cluster)[902340]: INFO: Resource agent invoked with: notify >Jul 14 17:15:37 ag(ag_cluster)[902340]: INFO: mssql_notify pre-start >Jul 14 17:15:37 ag(ag_cluster)[902340]: INFO: notify: 2023/07/14 17:15:37 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:15:37 ag(ag_cluster)[902340]: INFO: notify: 2023/07/14 17:15:37 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:15:37 ag(ag_cluster)[902340]: INFO: notify: 2023/07/14 17:15:37 Replica is RESOLVING (0) >Jul 14 17:15:37 ag(ag_cluster)[902340]: INFO: notify: 2023/07/14 17:15:37 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:15:37 ag(ag_cluster)[902340]: INFO: notify: 2023/07/14 17:15:37 Sequence number is 17F:F (1644972474383) >Jul 14 17:15:37 ag(ag_cluster)[902340]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:15:42 ag(ag_cluster)[902340]: INFO: ag_cluster notify : 0 >Jul 14 17:15:48 ag(ag_cluster)[902442]: INFO: mssql_validate >Jul 14 17:15:48 ag(ag_cluster)[902442]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:15:48 ag(ag_cluster)[902442]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:15:48 ag(ag_cluster)[902442]: INFO: Resource agent invoked with: notify >Jul 14 17:15:48 ag(ag_cluster)[902442]: INFO: mssql_notify post-start >Jul 14 17:15:48 ag(ag_cluster)[902442]: INFO: ag_cluster notify : 0 >Jul 14 17:15:48 ag(ag_cluster)[902488]: INFO: mssql_validate >Jul 14 17:15:48 ag(ag_cluster)[902488]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:15:48 ag(ag_cluster)[902488]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:15:48 ag(ag_cluster)[902488]: INFO: Resource agent invoked with: notify >Jul 14 17:15:48 ag(ag_cluster)[902488]: INFO: mssql_notify pre-start >Jul 14 17:15:48 ag(ag_cluster)[902488]: INFO: notify: 2023/07/14 17:15:48 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:15:48 ag(ag_cluster)[902488]: INFO: notify: 2023/07/14 17:15:48 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:15:48 ag(ag_cluster)[902488]: INFO: notify: 2023/07/14 17:15:48 Replica is RESOLVING (0) >Jul 14 17:15:48 ag(ag_cluster)[902488]: INFO: notify: 2023/07/14 17:15:48 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:15:48 ag(ag_cluster)[902488]: INFO: notify: 2023/07/14 17:15:48 Sequence number is 17F:F (1644972474383) >Jul 14 17:15:48 ag(ag_cluster)[902488]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:15:53 ag(ag_cluster)[902488]: INFO: ag_cluster notify : 0 >Jul 14 17:15:53 ag(ag_cluster)[902587]: INFO: mssql_validate >Jul 14 17:15:53 ag(ag_cluster)[902587]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:15:53 ag(ag_cluster)[902587]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:15:53 ag(ag_cluster)[902587]: INFO: Resource agent invoked with: notify >Jul 14 17:15:53 ag(ag_cluster)[902587]: INFO: mssql_notify pre-start >Jul 14 17:15:53 ag(ag_cluster)[902587]: INFO: notify: 2023/07/14 17:15:53 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:15:53 ag(ag_cluster)[902587]: INFO: notify: 2023/07/14 17:15:53 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:15:54 ag(ag_cluster)[902587]: INFO: notify: 2023/07/14 17:15:53 Replica is RESOLVING (0) >Jul 14 17:15:54 ag(ag_cluster)[902587]: INFO: notify: 2023/07/14 17:15:53 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:15:54 ag(ag_cluster)[902587]: INFO: notify: 2023/07/14 17:15:53 Sequence number is 17F:F (1644972474383) >Jul 14 17:15:54 ag(ag_cluster)[902587]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:15:59 ag(ag_cluster)[902587]: INFO: ag_cluster notify : 0 >Jul 14 17:15:59 ag(ag_cluster)[902686]: INFO: mssql_validate >Jul 14 17:15:59 ag(ag_cluster)[902686]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:15:59 ag(ag_cluster)[902686]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:15:59 ag(ag_cluster)[902686]: INFO: Resource agent invoked with: notify >Jul 14 17:15:59 ag(ag_cluster)[902686]: INFO: mssql_notify post-start >Jul 14 17:15:59 ag(ag_cluster)[902686]: INFO: ag_cluster notify : 0 >Jul 14 17:15:59 ag(ag_cluster)[902732]: INFO: mssql_validate >Jul 14 17:15:59 ag(ag_cluster)[902732]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:15:59 ag(ag_cluster)[902732]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:15:59 ag(ag_cluster)[902732]: INFO: Resource agent invoked with: notify >Jul 14 17:15:59 ag(ag_cluster)[902732]: INFO: mssql_notify pre-stop >Jul 14 17:15:59 ag(ag_cluster)[902732]: INFO: ag_cluster notify : 0 >Jul 14 17:15:59 ag(ag_cluster)[902777]: INFO: mssql_validate >Jul 14 17:15:59 ag(ag_cluster)[902777]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:15:59 ag(ag_cluster)[902777]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:15:59 ag(ag_cluster)[902777]: INFO: Resource agent invoked with: notify >Jul 14 17:15:59 ag(ag_cluster)[902777]: INFO: mssql_notify post-stop >Jul 14 17:15:59 ag(ag_cluster)[902777]: INFO: notify: 2023/07/14 17:15:59 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-stop]; connection-timeout [30]; health-threshold [3]; action [post-stop] >Jul 14 17:15:59 ag(ag_cluster)[902777]: INFO: notify: 2023/07/14 17:15:59 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:15:59 ag(ag_cluster)[902777]: INFO: notify: 2023/07/14 17:15:59 Replica is RESOLVING (0) >Jul 14 17:15:59 ag(ag_cluster)[902777]: INFO: ag_cluster notify : 0 >Jul 14 17:15:59 ag(ag_cluster)[902854]: INFO: mssql_validate >Jul 14 17:15:59 ag(ag_cluster)[902854]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:15:59 ag(ag_cluster)[902854]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:15:59 ag(ag_cluster)[902854]: INFO: Resource agent invoked with: notify >Jul 14 17:15:59 ag(ag_cluster)[902854]: INFO: mssql_notify pre-promote >Jul 14 17:15:59 ag(ag_cluster)[902854]: INFO: notify: 2023/07/14 17:15:59 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-promote]; connection-timeout [30]; health-threshold [3]; action [pre-promote] >Jul 14 17:15:59 ag(ag_cluster)[902854]: INFO: notify: 2023/07/14 17:15:59 From RetryExecute - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:15:59 ag(ag_cluster)[902854]: INFO: notify: 2023/07/14 17:15:59 Connected to the instance at localhost:1433 >Jul 14 17:16:04 ag(ag_cluster)[902854]: INFO: notify: 2023/07/14 17:16:04 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:16:04 ag(ag_cluster)[902854]: INFO: notify: 2023/07/14 17:16:04 Sequence number is 17F:F (1644972474383) >Jul 14 17:16:04 ag(ag_cluster)[902854]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:16:09 ag(ag_cluster)[902854]: INFO: ag_cluster notify : 0 >Jul 14 17:16:09 ag(ag_cluster)[902959]: INFO: mssql_validate >Jul 14 17:16:09 ag(ag_cluster)[902959]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:16:10 ag(ag_cluster)[902959]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:16:10 ag(ag_cluster)[902959]: INFO: Resource agent invoked with: notify >Jul 14 17:16:10 ag(ag_cluster)[902959]: INFO: mssql_notify pre-promote >Jul 14 17:16:10 ag(ag_cluster)[902959]: INFO: notify: 2023/07/14 17:16:10 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-promote]; connection-timeout [30]; health-threshold [3]; action [pre-promote] >Jul 14 17:16:10 ag(ag_cluster)[902959]: INFO: notify: 2023/07/14 17:16:10 From RetryExecute - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:16:10 ag(ag_cluster)[902959]: INFO: notify: 2023/07/14 17:16:10 Connected to the instance at localhost:1433 >Jul 14 17:16:15 ag(ag_cluster)[902959]: INFO: notify: 2023/07/14 17:16:15 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:16:15 ag(ag_cluster)[902959]: INFO: notify: 2023/07/14 17:16:15 Sequence number is 17F:F (1644972474383) >Jul 14 17:16:15 ag(ag_cluster)[902959]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:16:20 ag(ag_cluster)[902959]: INFO: ag_cluster notify : 0 >Jul 14 17:16:25 ag(ag_cluster)[903069]: INFO: mssql_validate >Jul 14 17:16:25 ag(ag_cluster)[903069]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:16:25 ag(ag_cluster)[903069]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:16:25 ag(ag_cluster)[903069]: INFO: Resource agent invoked with: notify >Jul 14 17:16:25 ag(ag_cluster)[903069]: INFO: mssql_notify post-promote >Jul 14 17:16:25 ag(ag_cluster)[903069]: INFO: notify: 2023/07/14 17:16:25 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-promote]; connection-timeout [30]; health-threshold [3]; action [post-promote] >Jul 14 17:16:25 ag(ag_cluster)[903069]: INFO: notify: 2023/07/14 17:16:25 [DEBUG] AG Helper PostPromote Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:16:25 ag(ag_cluster)[903069]: INFO: notify: 2023/07/14 17:16:25 Replica is RESOLVING (0) >Jul 14 17:16:25 ag(ag_cluster)[903069]: INFO: ag_cluster notify : 0 >Jul 14 17:16:25 ag(ag_cluster)[903145]: INFO: mssql_validate >Jul 14 17:16:25 ag(ag_cluster)[903145]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:16:25 ag(ag_cluster)[903145]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:16:25 ag(ag_cluster)[903145]: INFO: Resource agent invoked with: monitor >Jul 14 17:16:25 ag(ag_cluster)[903145]: INFO: mssql_monitor >Jul 14 17:16:26 ag(ag_cluster)[903145]: INFO: monitor: 2023/07/14 17:16:26 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:16:26 ag(ag_cluster)[903145]: INFO: monitor: 2023/07/14 17:16:26 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:16:26 ag(ag_cluster)[903145]: INFO: monitor: 2023/07/14 17:16:26 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:16:26 ag(ag_cluster)[903145]: INFO: monitor: 2023/07/14 17:16:26 Connected to the instance at localhost:1433 >Jul 14 17:16:31 ag(ag_cluster)[903145]: INFO: monitor: 2023/07/14 17:16:31 Monitor Caller is: monitor. >Jul 14 17:16:31 ag(ag_cluster)[903145]: INFO: monitor: 2023/07/14 17:16:31 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:16:31 ag(ag_cluster)[903145]: INFO: monitor: 2023/07/14 17:16:31 Replica is RESOLVING (0) >Jul 14 17:16:31 ag(ag_cluster)[903145]: INFO: monitor: 2023/07/14 17:16:31 Instance name is server3. >Jul 14 17:16:31 ag(ag_cluster)[903145]: INFO: monitor: 2023/07/14 17:16:31 Current master is server2. >Jul 14 17:16:31 ag(ag_cluster)[903145]: INFO: monitor: 2023/07/14 17:16:31 Setting the role to Secondary. >Jul 14 17:16:31 ag(ag_cluster)[903145]: INFO: monitor: 2023/07/14 17:16:31 Setting replica to SECONDARY role... >Jul 14 17:16:31 ag(ag_cluster)[903145]: INFO: monitor: 2023/07/14 17:16:31 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:16:31 ag(ag_cluster)[903145]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:16:31 ag(ag_cluster)[903145]: INFO: lease_expiry after monitor update: >Jul 14 17:16:31 ag(ag_cluster)[903145]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:16:31 ag(ag_cluster)[903145]: INFO: ag_cluster monitor : 0 >Jul 14 17:16:31 ag(ag_cluster)[903326]: INFO: mssql_validate >Jul 14 17:16:31 ag(ag_cluster)[903326]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:16:31 ag(ag_cluster)[903326]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:16:31 ag(ag_cluster)[903326]: INFO: Resource agent invoked with: monitor >Jul 14 17:16:31 ag(ag_cluster)[903326]: INFO: mssql_monitor >Jul 14 17:16:31 ag(ag_cluster)[903326]: INFO: monitor: 2023/07/14 17:16:31 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:16:31 ag(ag_cluster)[903326]: INFO: monitor: 2023/07/14 17:16:31 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:16:31 ag(ag_cluster)[903326]: INFO: monitor: 2023/07/14 17:16:31 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:16:31 ag(ag_cluster)[903326]: INFO: monitor: 2023/07/14 17:16:31 Connected to the instance at localhost:1433 >Jul 14 17:16:36 ag(ag_cluster)[903326]: INFO: monitor: 2023/07/14 17:16:36 Monitor Caller is: monitor. >Jul 14 17:16:36 ag(ag_cluster)[903326]: INFO: monitor: 2023/07/14 17:16:36 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:16:36 ag(ag_cluster)[903326]: INFO: monitor: 2023/07/14 17:16:36 Replica is RESOLVING (0) >Jul 14 17:16:36 ag(ag_cluster)[903326]: INFO: monitor: 2023/07/14 17:16:36 Instance name is server3. >Jul 14 17:16:36 ag(ag_cluster)[903326]: INFO: monitor: 2023/07/14 17:16:36 Current master is server2. >Jul 14 17:16:36 ag(ag_cluster)[903326]: INFO: monitor: 2023/07/14 17:16:36 Setting the role to Secondary. >Jul 14 17:16:36 ag(ag_cluster)[903326]: INFO: monitor: 2023/07/14 17:16:36 Setting replica to SECONDARY role... >Jul 14 17:16:36 ag(ag_cluster)[903326]: INFO: monitor: 2023/07/14 17:16:36 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:16:36 ag(ag_cluster)[903326]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:16:36 ag(ag_cluster)[903326]: INFO: lease_expiry after monitor update: >Jul 14 17:16:36 ag(ag_cluster)[903326]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:16:36 ag(ag_cluster)[903326]: INFO: ag_cluster monitor : 0 >Jul 14 17:16:43 ag(ag_cluster)[903499]: INFO: mssql_validate >Jul 14 17:16:43 ag(ag_cluster)[903499]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:16:43 ag(ag_cluster)[903499]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:16:43 ag(ag_cluster)[903499]: INFO: Resource agent invoked with: monitor >Jul 14 17:16:43 ag(ag_cluster)[903499]: INFO: mssql_monitor >Jul 14 17:16:43 ag(ag_cluster)[903499]: INFO: monitor: 2023/07/14 17:16:43 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:16:43 ag(ag_cluster)[903499]: INFO: monitor: 2023/07/14 17:16:43 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:16:43 ag(ag_cluster)[903499]: INFO: monitor: 2023/07/14 17:16:43 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:16:43 ag(ag_cluster)[903499]: INFO: monitor: 2023/07/14 17:16:43 Connected to the instance at localhost:1433 >Jul 14 17:16:48 ag(ag_cluster)[903499]: INFO: monitor: 2023/07/14 17:16:48 Monitor Caller is: monitor. >Jul 14 17:16:48 ag(ag_cluster)[903499]: INFO: monitor: 2023/07/14 17:16:48 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:16:48 ag(ag_cluster)[903499]: INFO: monitor: 2023/07/14 17:16:48 Replica is RESOLVING (0) >Jul 14 17:16:48 ag(ag_cluster)[903499]: INFO: monitor: 2023/07/14 17:16:48 Instance name is server3. >Jul 14 17:16:48 ag(ag_cluster)[903499]: INFO: monitor: 2023/07/14 17:16:48 Current master is server2. >Jul 14 17:16:48 ag(ag_cluster)[903499]: INFO: monitor: 2023/07/14 17:16:48 Setting the role to Secondary. >Jul 14 17:16:48 ag(ag_cluster)[903499]: INFO: monitor: 2023/07/14 17:16:48 Setting replica to SECONDARY role... >Jul 14 17:16:48 ag(ag_cluster)[903499]: INFO: monitor: 2023/07/14 17:16:48 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:16:48 ag(ag_cluster)[903499]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:16:48 ag(ag_cluster)[903499]: INFO: lease_expiry after monitor update: >Jul 14 17:16:48 ag(ag_cluster)[903499]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:16:48 ag(ag_cluster)[903499]: INFO: ag_cluster monitor : 0 >Jul 14 17:16:48 ag(ag_cluster)[903670]: INFO: mssql_validate >Jul 14 17:16:48 ag(ag_cluster)[903670]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:16:48 ag(ag_cluster)[903670]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:16:48 ag(ag_cluster)[903670]: INFO: Resource agent invoked with: monitor >Jul 14 17:16:48 ag(ag_cluster)[903670]: INFO: mssql_monitor >Jul 14 17:16:48 ag(ag_cluster)[903670]: INFO: monitor: 2023/07/14 17:16:48 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:16:48 ag(ag_cluster)[903670]: INFO: monitor: 2023/07/14 17:16:48 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:16:48 ag(ag_cluster)[903670]: INFO: monitor: 2023/07/14 17:16:48 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:16:48 ag(ag_cluster)[903670]: INFO: monitor: 2023/07/14 17:16:48 Connected to the instance at localhost:1433 >Jul 14 17:16:53 ag(ag_cluster)[903670]: INFO: monitor: 2023/07/14 17:16:53 Monitor Caller is: monitor. >Jul 14 17:16:53 ag(ag_cluster)[903670]: INFO: monitor: 2023/07/14 17:16:53 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:16:53 ag(ag_cluster)[903670]: INFO: monitor: 2023/07/14 17:16:53 Replica is RESOLVING (0) >Jul 14 17:16:53 ag(ag_cluster)[903670]: INFO: monitor: 2023/07/14 17:16:53 Instance name is server3. >Jul 14 17:16:53 ag(ag_cluster)[903670]: INFO: monitor: 2023/07/14 17:16:53 Current master is server2. >Jul 14 17:16:53 ag(ag_cluster)[903670]: INFO: monitor: 2023/07/14 17:16:53 Setting the role to Secondary. >Jul 14 17:16:54 ag(ag_cluster)[903670]: INFO: monitor: 2023/07/14 17:16:53 Setting replica to SECONDARY role... >Jul 14 17:16:54 ag(ag_cluster)[903670]: INFO: monitor: 2023/07/14 17:16:53 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:16:54 ag(ag_cluster)[903670]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:16:54 ag(ag_cluster)[903670]: INFO: lease_expiry after monitor update: >Jul 14 17:16:54 ag(ag_cluster)[903670]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:16:54 ag(ag_cluster)[903670]: INFO: ag_cluster monitor : 0 >Jul 14 17:17:00 ag(ag_cluster)[903843]: INFO: mssql_validate >Jul 14 17:17:00 ag(ag_cluster)[903843]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:17:00 ag(ag_cluster)[903843]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:17:00 ag(ag_cluster)[903843]: INFO: Resource agent invoked with: monitor >Jul 14 17:17:00 ag(ag_cluster)[903843]: INFO: mssql_monitor >Jul 14 17:17:00 ag(ag_cluster)[903843]: INFO: monitor: 2023/07/14 17:17:00 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:17:00 ag(ag_cluster)[903843]: INFO: monitor: 2023/07/14 17:17:00 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:17:00 ag(ag_cluster)[903843]: INFO: monitor: 2023/07/14 17:17:00 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:17:00 ag(ag_cluster)[903843]: INFO: monitor: 2023/07/14 17:17:00 Connected to the instance at localhost:1433 >Jul 14 17:17:05 ag(ag_cluster)[903843]: INFO: monitor: 2023/07/14 17:17:05 Monitor Caller is: monitor. >Jul 14 17:17:06 ag(ag_cluster)[903843]: INFO: monitor: 2023/07/14 17:17:05 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:17:06 ag(ag_cluster)[903843]: INFO: monitor: 2023/07/14 17:17:05 Replica is RESOLVING (0) >Jul 14 17:17:06 ag(ag_cluster)[903843]: INFO: monitor: 2023/07/14 17:17:05 Instance name is server3. >Jul 14 17:17:06 ag(ag_cluster)[903843]: INFO: monitor: 2023/07/14 17:17:05 Current master is server2. >Jul 14 17:17:06 ag(ag_cluster)[903843]: INFO: monitor: 2023/07/14 17:17:05 Setting the role to Secondary. >Jul 14 17:17:06 ag(ag_cluster)[903843]: INFO: monitor: 2023/07/14 17:17:05 Setting replica to SECONDARY role... >Jul 14 17:17:06 ag(ag_cluster)[903843]: INFO: monitor: 2023/07/14 17:17:05 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:17:06 ag(ag_cluster)[903843]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:17:06 ag(ag_cluster)[903843]: INFO: lease_expiry after monitor update: >Jul 14 17:17:06 ag(ag_cluster)[903843]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:17:06 ag(ag_cluster)[903843]: INFO: ag_cluster monitor : 0 >Jul 14 17:17:06 ag(ag_cluster)[904015]: INFO: mssql_validate >Jul 14 17:17:06 ag(ag_cluster)[904015]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:17:06 ag(ag_cluster)[904015]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:17:06 ag(ag_cluster)[904015]: INFO: Resource agent invoked with: notify >Jul 14 17:17:06 ag(ag_cluster)[904015]: INFO: mssql_notify pre-start >Jul 14 17:17:06 ag(ag_cluster)[904015]: INFO: notify: 2023/07/14 17:17:06 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:17:06 ag(ag_cluster)[904015]: INFO: notify: 2023/07/14 17:17:06 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:17:06 ag(ag_cluster)[904015]: INFO: notify: 2023/07/14 17:17:06 Replica is RESOLVING (0) >Jul 14 17:17:06 ag(ag_cluster)[904015]: INFO: notify: 2023/07/14 17:17:06 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:17:06 ag(ag_cluster)[904015]: INFO: notify: 2023/07/14 17:17:06 Sequence number is 17F:F (1644972474383) >Jul 14 17:17:06 ag(ag_cluster)[904015]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:17:11 ag(ag_cluster)[904015]: INFO: ag_cluster notify : 0 >Jul 14 17:17:11 ag(ag_cluster)[904114]: INFO: mssql_validate >Jul 14 17:17:11 ag(ag_cluster)[904114]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:17:11 ag(ag_cluster)[904114]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:17:11 ag(ag_cluster)[904114]: INFO: Resource agent invoked with: monitor >Jul 14 17:17:11 ag(ag_cluster)[904114]: INFO: mssql_monitor >Jul 14 17:17:11 ag(ag_cluster)[904114]: INFO: monitor: 2023/07/14 17:17:11 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:17:11 ag(ag_cluster)[904114]: INFO: monitor: 2023/07/14 17:17:11 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:17:11 ag(ag_cluster)[904114]: INFO: monitor: 2023/07/14 17:17:11 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:17:11 ag(ag_cluster)[904114]: INFO: monitor: 2023/07/14 17:17:11 Connected to the instance at localhost:1433 >Jul 14 17:17:16 ag(ag_cluster)[904114]: INFO: monitor: 2023/07/14 17:17:16 Monitor Caller is: monitor. >Jul 14 17:17:16 ag(ag_cluster)[904114]: INFO: monitor: 2023/07/14 17:17:16 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:17:16 ag(ag_cluster)[904114]: INFO: monitor: 2023/07/14 17:17:16 Replica is RESOLVING (0) >Jul 14 17:17:16 ag(ag_cluster)[904114]: INFO: monitor: 2023/07/14 17:17:16 Instance name is server3. >Jul 14 17:17:16 ag(ag_cluster)[904114]: INFO: monitor: 2023/07/14 17:17:16 Current master is server2. >Jul 14 17:17:16 ag(ag_cluster)[904114]: INFO: monitor: 2023/07/14 17:17:16 Setting the role to Secondary. >Jul 14 17:17:16 ag(ag_cluster)[904114]: INFO: monitor: 2023/07/14 17:17:16 Setting replica to SECONDARY role... >Jul 14 17:17:16 ag(ag_cluster)[904114]: INFO: monitor: 2023/07/14 17:17:16 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:17:16 ag(ag_cluster)[904114]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:17:16 ag(ag_cluster)[904114]: INFO: lease_expiry after monitor update: >Jul 14 17:17:16 ag(ag_cluster)[904114]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:17:16 ag(ag_cluster)[904114]: INFO: ag_cluster monitor : 0 >Jul 14 17:17:16 ag(ag_cluster)[904291]: INFO: mssql_validate >Jul 14 17:17:16 ag(ag_cluster)[904291]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:17:16 ag(ag_cluster)[904291]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:17:16 ag(ag_cluster)[904291]: INFO: Resource agent invoked with: notify >Jul 14 17:17:16 ag(ag_cluster)[904291]: INFO: mssql_notify pre-start >Jul 14 17:17:16 ag(ag_cluster)[904291]: INFO: notify: 2023/07/14 17:17:16 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:17:16 ag(ag_cluster)[904291]: INFO: notify: 2023/07/14 17:17:16 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:17:16 ag(ag_cluster)[904291]: INFO: notify: 2023/07/14 17:17:16 Replica is RESOLVING (0) >Jul 14 17:17:16 ag(ag_cluster)[904291]: INFO: notify: 2023/07/14 17:17:16 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:17:16 ag(ag_cluster)[904291]: INFO: notify: 2023/07/14 17:17:16 Sequence number is 17F:F (1644972474383) >Jul 14 17:17:16 ag(ag_cluster)[904291]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:17:21 ag(ag_cluster)[904291]: INFO: ag_cluster notify : 0 >Jul 14 17:17:21 ag(ag_cluster)[904390]: INFO: mssql_validate >Jul 14 17:17:22 ag(ag_cluster)[904390]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:17:22 ag(ag_cluster)[904390]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:17:22 ag(ag_cluster)[904390]: INFO: Resource agent invoked with: monitor >Jul 14 17:17:22 ag(ag_cluster)[904390]: INFO: mssql_monitor >Jul 14 17:17:22 ag(ag_cluster)[904390]: INFO: monitor: 2023/07/14 17:17:22 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:17:22 ag(ag_cluster)[904390]: INFO: monitor: 2023/07/14 17:17:22 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:17:22 ag(ag_cluster)[904390]: INFO: monitor: 2023/07/14 17:17:22 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:17:22 ag(ag_cluster)[904390]: INFO: monitor: 2023/07/14 17:17:22 Connected to the instance at localhost:1433 >Jul 14 17:17:27 ag(ag_cluster)[904390]: INFO: monitor: 2023/07/14 17:17:27 Monitor Caller is: monitor. >Jul 14 17:17:27 ag(ag_cluster)[904390]: INFO: monitor: 2023/07/14 17:17:27 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:17:27 ag(ag_cluster)[904390]: INFO: monitor: 2023/07/14 17:17:27 Replica is RESOLVING (0) >Jul 14 17:17:27 ag(ag_cluster)[904390]: INFO: monitor: 2023/07/14 17:17:27 Instance name is server3. >Jul 14 17:17:27 ag(ag_cluster)[904390]: INFO: monitor: 2023/07/14 17:17:27 Current master is server2. >Jul 14 17:17:27 ag(ag_cluster)[904390]: INFO: monitor: 2023/07/14 17:17:27 Setting the role to Secondary. >Jul 14 17:17:27 ag(ag_cluster)[904390]: INFO: monitor: 2023/07/14 17:17:27 Setting replica to SECONDARY role... >Jul 14 17:17:27 ag(ag_cluster)[904390]: INFO: monitor: 2023/07/14 17:17:27 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:17:27 ag(ag_cluster)[904390]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:17:27 ag(ag_cluster)[904390]: INFO: lease_expiry after monitor update: >Jul 14 17:17:27 ag(ag_cluster)[904390]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:17:27 ag(ag_cluster)[904390]: INFO: ag_cluster monitor : 0 >Jul 14 17:17:27 ag(ag_cluster)[904563]: INFO: mssql_validate >Jul 14 17:17:27 ag(ag_cluster)[904563]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:17:27 ag(ag_cluster)[904563]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:17:27 ag(ag_cluster)[904563]: INFO: Resource agent invoked with: notify >Jul 14 17:17:27 ag(ag_cluster)[904563]: INFO: mssql_notify post-start >Jul 14 17:17:27 ag(ag_cluster)[904563]: INFO: ag_cluster notify : 0 >Jul 14 17:17:27 ag(ag_cluster)[904609]: INFO: mssql_validate >Jul 14 17:17:27 ag(ag_cluster)[904609]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:17:27 ag(ag_cluster)[904609]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:17:27 ag(ag_cluster)[904609]: INFO: Resource agent invoked with: monitor >Jul 14 17:17:27 ag(ag_cluster)[904609]: INFO: mssql_monitor >Jul 14 17:17:27 ag(ag_cluster)[904609]: INFO: monitor: 2023/07/14 17:17:27 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:17:27 ag(ag_cluster)[904609]: INFO: monitor: 2023/07/14 17:17:27 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:17:27 ag(ag_cluster)[904609]: INFO: monitor: 2023/07/14 17:17:27 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:17:27 ag(ag_cluster)[904609]: INFO: monitor: 2023/07/14 17:17:27 Connected to the instance at localhost:1433 >Jul 14 17:17:32 ag(ag_cluster)[904609]: INFO: monitor: 2023/07/14 17:17:32 Monitor Caller is: monitor. >Jul 14 17:17:32 ag(ag_cluster)[904609]: INFO: monitor: 2023/07/14 17:17:32 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:17:32 ag(ag_cluster)[904609]: INFO: monitor: 2023/07/14 17:17:32 Replica is RESOLVING (0) >Jul 14 17:17:32 ag(ag_cluster)[904609]: INFO: monitor: 2023/07/14 17:17:32 Instance name is server3. >Jul 14 17:17:32 ag(ag_cluster)[904609]: INFO: monitor: 2023/07/14 17:17:32 Current master is server2. >Jul 14 17:17:32 ag(ag_cluster)[904609]: INFO: monitor: 2023/07/14 17:17:32 Setting the role to Secondary. >Jul 14 17:17:32 ag(ag_cluster)[904609]: INFO: monitor: 2023/07/14 17:17:32 Setting replica to SECONDARY role... >Jul 14 17:17:32 ag(ag_cluster)[904609]: INFO: monitor: 2023/07/14 17:17:32 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:17:32 ag(ag_cluster)[904609]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:17:32 ag(ag_cluster)[904609]: INFO: lease_expiry after monitor update: >Jul 14 17:17:32 ag(ag_cluster)[904609]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:17:32 ag(ag_cluster)[904609]: INFO: ag_cluster monitor : 0 >Jul 14 17:17:32 ag(ag_cluster)[904790]: INFO: mssql_validate >Jul 14 17:17:32 ag(ag_cluster)[904790]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:17:32 ag(ag_cluster)[904790]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:17:32 ag(ag_cluster)[904790]: INFO: Resource agent invoked with: notify >Jul 14 17:17:33 ag(ag_cluster)[904790]: INFO: mssql_notify pre-stop >Jul 14 17:17:33 ag(ag_cluster)[904790]: INFO: ag_cluster notify : 0 >Jul 14 17:17:33 ag(ag_cluster)[904835]: INFO: mssql_validate >Jul 14 17:17:33 ag(ag_cluster)[904835]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:17:33 ag(ag_cluster)[904835]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:17:33 ag(ag_cluster)[904835]: INFO: Resource agent invoked with: notify >Jul 14 17:17:33 ag(ag_cluster)[904835]: INFO: mssql_notify post-stop >Jul 14 17:17:33 ag(ag_cluster)[904835]: INFO: notify: 2023/07/14 17:17:33 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-stop]; connection-timeout [30]; health-threshold [3]; action [post-stop] >Jul 14 17:17:33 ag(ag_cluster)[904835]: INFO: notify: 2023/07/14 17:17:33 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:17:33 ag(ag_cluster)[904835]: INFO: notify: 2023/07/14 17:17:33 Replica is RESOLVING (0) >Jul 14 17:17:33 ag(ag_cluster)[904835]: INFO: ag_cluster notify : 0 >Jul 14 17:17:39 ag(ag_cluster)[904910]: INFO: mssql_validate >Jul 14 17:17:39 ag(ag_cluster)[904910]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:17:39 ag(ag_cluster)[904910]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:17:39 ag(ag_cluster)[904910]: INFO: Resource agent invoked with: monitor >Jul 14 17:17:39 ag(ag_cluster)[904910]: INFO: mssql_monitor >Jul 14 17:17:39 ag(ag_cluster)[904910]: INFO: monitor: 2023/07/14 17:17:39 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:17:39 ag(ag_cluster)[904910]: INFO: monitor: 2023/07/14 17:17:39 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:17:39 ag(ag_cluster)[904910]: INFO: monitor: 2023/07/14 17:17:39 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:17:39 ag(ag_cluster)[904910]: INFO: monitor: 2023/07/14 17:17:39 Connected to the instance at localhost:1433 >Jul 14 17:17:44 ag(ag_cluster)[904910]: INFO: monitor: 2023/07/14 17:17:44 Monitor Caller is: monitor. >Jul 14 17:17:44 ag(ag_cluster)[904910]: INFO: monitor: 2023/07/14 17:17:44 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:17:44 ag(ag_cluster)[904910]: INFO: monitor: 2023/07/14 17:17:44 Replica is RESOLVING (0) >Jul 14 17:17:44 ag(ag_cluster)[904910]: INFO: monitor: 2023/07/14 17:17:44 Instance name is server3. >Jul 14 17:17:44 ag(ag_cluster)[904910]: INFO: monitor: 2023/07/14 17:17:44 Current master is server2. >Jul 14 17:17:44 ag(ag_cluster)[904910]: INFO: monitor: 2023/07/14 17:17:44 Setting the role to Secondary. >Jul 14 17:17:44 ag(ag_cluster)[904910]: INFO: monitor: 2023/07/14 17:17:44 Setting replica to SECONDARY role... >Jul 14 17:17:44 ag(ag_cluster)[904910]: INFO: monitor: 2023/07/14 17:17:44 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:17:44 ag(ag_cluster)[904910]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:17:44 ag(ag_cluster)[904910]: INFO: lease_expiry after monitor update: >Jul 14 17:17:44 ag(ag_cluster)[904910]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:17:44 ag(ag_cluster)[904910]: INFO: ag_cluster monitor : 0 >Jul 14 17:17:44 ag(ag_cluster)[905082]: INFO: mssql_validate >Jul 14 17:17:44 ag(ag_cluster)[905082]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:17:44 ag(ag_cluster)[905082]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:17:44 ag(ag_cluster)[905082]: INFO: Resource agent invoked with: monitor >Jul 14 17:17:44 ag(ag_cluster)[905082]: INFO: mssql_monitor >Jul 14 17:17:45 ag(ag_cluster)[905082]: INFO: monitor: 2023/07/14 17:17:45 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:17:45 ag(ag_cluster)[905082]: INFO: monitor: 2023/07/14 17:17:45 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:17:45 ag(ag_cluster)[905082]: INFO: monitor: 2023/07/14 17:17:45 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:17:45 ag(ag_cluster)[905082]: INFO: monitor: 2023/07/14 17:17:45 Connected to the instance at localhost:1433 >Jul 14 17:17:50 ag(ag_cluster)[905082]: INFO: monitor: 2023/07/14 17:17:50 Monitor Caller is: monitor. >Jul 14 17:17:50 ag(ag_cluster)[905082]: INFO: monitor: 2023/07/14 17:17:50 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:17:50 ag(ag_cluster)[905082]: INFO: monitor: 2023/07/14 17:17:50 Replica is RESOLVING (0) >Jul 14 17:17:50 ag(ag_cluster)[905082]: INFO: monitor: 2023/07/14 17:17:50 Instance name is server3. >Jul 14 17:17:50 ag(ag_cluster)[905082]: INFO: monitor: 2023/07/14 17:17:50 Current master is server2. >Jul 14 17:17:50 ag(ag_cluster)[905082]: INFO: monitor: 2023/07/14 17:17:50 Setting the role to Secondary. >Jul 14 17:17:50 ag(ag_cluster)[905082]: INFO: monitor: 2023/07/14 17:17:50 Setting replica to SECONDARY role... >Jul 14 17:17:50 ag(ag_cluster)[905082]: INFO: monitor: 2023/07/14 17:17:50 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:17:50 ag(ag_cluster)[905082]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:17:50 ag(ag_cluster)[905082]: INFO: lease_expiry after monitor update: >Jul 14 17:17:50 ag(ag_cluster)[905082]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:17:50 ag(ag_cluster)[905082]: INFO: ag_cluster monitor : 0 >Jul 14 17:17:55 ag(ag_cluster)[905254]: INFO: mssql_validate >Jul 14 17:17:55 ag(ag_cluster)[905254]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:17:55 ag(ag_cluster)[905254]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:17:55 ag(ag_cluster)[905254]: INFO: Resource agent invoked with: notify >Jul 14 17:17:55 ag(ag_cluster)[905254]: INFO: mssql_notify pre-demote >Jul 14 17:17:55 ag(ag_cluster)[905254]: INFO: ag_cluster notify : 0 >Jul 14 17:17:55 ag(ag_cluster)[905299]: INFO: mssql_validate >Jul 14 17:17:55 ag(ag_cluster)[905299]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:17:55 ag(ag_cluster)[905299]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:17:55 ag(ag_cluster)[905299]: INFO: Resource agent invoked with: notify >Jul 14 17:17:55 ag(ag_cluster)[905299]: INFO: mssql_notify post-demote >Jul 14 17:17:55 ag(ag_cluster)[905299]: INFO: ag_cluster notify : 0 >Jul 14 17:17:55 ag(ag_cluster)[905344]: INFO: mssql_validate >Jul 14 17:17:55 ag(ag_cluster)[905344]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:17:55 ag(ag_cluster)[905344]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:17:55 ag(ag_cluster)[905344]: INFO: Resource agent invoked with: notify >Jul 14 17:17:55 ag(ag_cluster)[905344]: INFO: mssql_notify pre-stop >Jul 14 17:17:55 ag(ag_cluster)[905344]: INFO: ag_cluster notify : 0 >Jul 14 17:17:55 ag(ag_cluster)[905389]: INFO: mssql_validate >Jul 14 17:17:55 ag(ag_cluster)[905389]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:17:55 ag(ag_cluster)[905389]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:17:55 ag(ag_cluster)[905389]: INFO: Resource agent invoked with: notify >Jul 14 17:17:55 ag(ag_cluster)[905389]: INFO: mssql_notify post-stop >Jul 14 17:17:55 ag(ag_cluster)[905389]: INFO: notify: 2023/07/14 17:17:55 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-stop]; connection-timeout [30]; health-threshold [3]; action [post-stop] >Jul 14 17:17:55 ag(ag_cluster)[905389]: INFO: notify: 2023/07/14 17:17:55 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:17:56 ag(ag_cluster)[905389]: INFO: notify: 2023/07/14 17:17:56 Replica is RESOLVING (0) >Jul 14 17:17:56 ag(ag_cluster)[905389]: INFO: ag_cluster notify : 0 >Jul 14 17:17:56 ag(ag_cluster)[905463]: INFO: mssql_validate >Jul 14 17:17:56 ag(ag_cluster)[905463]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:17:56 ag(ag_cluster)[905463]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:17:56 ag(ag_cluster)[905463]: INFO: Resource agent invoked with: notify >Jul 14 17:17:56 ag(ag_cluster)[905463]: INFO: mssql_notify pre-start >Jul 14 17:17:56 ag(ag_cluster)[905463]: INFO: notify: 2023/07/14 17:17:56 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:17:56 ag(ag_cluster)[905463]: INFO: notify: 2023/07/14 17:17:56 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:17:56 ag(ag_cluster)[905463]: INFO: notify: 2023/07/14 17:17:56 Replica is RESOLVING (0) >Jul 14 17:17:56 ag(ag_cluster)[905463]: INFO: notify: 2023/07/14 17:17:56 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:17:56 ag(ag_cluster)[905463]: INFO: notify: 2023/07/14 17:17:56 Sequence number is 17F:F (1644972474383) >Jul 14 17:17:56 ag(ag_cluster)[905463]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:18:01 ag(ag_cluster)[905463]: INFO: ag_cluster notify : 0 >Jul 14 17:18:01 ag(ag_cluster)[905562]: INFO: mssql_validate >Jul 14 17:18:01 ag(ag_cluster)[905562]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:18:01 ag(ag_cluster)[905562]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:18:01 ag(ag_cluster)[905562]: INFO: Resource agent invoked with: monitor >Jul 14 17:18:01 ag(ag_cluster)[905562]: INFO: mssql_monitor >Jul 14 17:18:01 ag(ag_cluster)[905562]: INFO: monitor: 2023/07/14 17:18:01 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:18:01 ag(ag_cluster)[905562]: INFO: monitor: 2023/07/14 17:18:01 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master []; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:18:01 ag(ag_cluster)[905562]: INFO: monitor: 2023/07/14 17:18:01 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:18:01 ag(ag_cluster)[905562]: INFO: monitor: 2023/07/14 17:18:01 Connected to the instance at localhost:1433 >Jul 14 17:18:06 ag(ag_cluster)[905562]: INFO: monitor: 2023/07/14 17:18:06 Monitor Caller is: monitor. >Jul 14 17:18:06 ag(ag_cluster)[905562]: INFO: monitor: 2023/07/14 17:18:06 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:18:06 ag(ag_cluster)[905562]: INFO: monitor: 2023/07/14 17:18:06 Replica is RESOLVING (0) >Jul 14 17:18:06 ag(ag_cluster)[905562]: INFO: monitor: 2023/07/14 17:18:06 Instance name is server3. >Jul 14 17:18:06 ag(ag_cluster)[905562]: INFO: monitor: 2023/07/14 17:18:06 Current master is . >Jul 14 17:18:06 ag(ag_cluster)[905562]: INFO: monitor: PROMOTION_SCORE: 10 >Jul 14 17:18:06 ag(ag_cluster)[905562]: INFO: lease_expiry after monitor update: >Jul 14 17:18:06 ag(ag_cluster)[905562]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:18:06 ag(ag_cluster)[905562]: INFO: ag_cluster monitor : 0 >Jul 14 17:18:06 ag(ag_cluster)[905716]: INFO: mssql_validate >Jul 14 17:18:06 ag(ag_cluster)[905716]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:18:06 ag(ag_cluster)[905716]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:18:06 ag(ag_cluster)[905716]: INFO: Resource agent invoked with: monitor >Jul 14 17:18:06 ag(ag_cluster)[905716]: INFO: mssql_monitor >Jul 14 17:18:06 ag(ag_cluster)[905716]: INFO: monitor: 2023/07/14 17:18:06 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:18:06 ag(ag_cluster)[905716]: INFO: monitor: 2023/07/14 17:18:06 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master []; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:18:07 ag(ag_cluster)[905716]: INFO: monitor: 2023/07/14 17:18:06 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:18:07 ag(ag_cluster)[905716]: INFO: monitor: 2023/07/14 17:18:06 Connected to the instance at localhost:1433 >Jul 14 17:18:12 ag(ag_cluster)[905716]: INFO: monitor: 2023/07/14 17:18:12 Monitor Caller is: monitor. >Jul 14 17:18:12 ag(ag_cluster)[905716]: INFO: monitor: 2023/07/14 17:18:12 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:18:12 ag(ag_cluster)[905716]: INFO: monitor: 2023/07/14 17:18:12 Replica is RESOLVING (0) >Jul 14 17:18:12 ag(ag_cluster)[905716]: INFO: monitor: 2023/07/14 17:18:12 Instance name is server3. >Jul 14 17:18:12 ag(ag_cluster)[905716]: INFO: monitor: 2023/07/14 17:18:12 Current master is . >Jul 14 17:18:12 ag(ag_cluster)[905716]: INFO: monitor: PROMOTION_SCORE: 10 >Jul 14 17:18:12 ag(ag_cluster)[905716]: INFO: lease_expiry after monitor update: >Jul 14 17:18:12 ag(ag_cluster)[905716]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:18:12 ag(ag_cluster)[905716]: INFO: ag_cluster monitor : 0 >Jul 14 17:18:12 ag(ag_cluster)[905871]: INFO: mssql_validate >Jul 14 17:18:12 ag(ag_cluster)[905871]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:18:12 ag(ag_cluster)[905871]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:18:12 ag(ag_cluster)[905871]: INFO: Resource agent invoked with: notify >Jul 14 17:18:12 ag(ag_cluster)[905871]: INFO: mssql_notify pre-start >Jul 14 17:18:12 ag(ag_cluster)[905871]: INFO: notify: 2023/07/14 17:18:12 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:18:12 ag(ag_cluster)[905871]: INFO: notify: 2023/07/14 17:18:12 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:18:12 ag(ag_cluster)[905871]: INFO: notify: 2023/07/14 17:18:12 Replica is RESOLVING (0) >Jul 14 17:18:12 ag(ag_cluster)[905871]: INFO: notify: 2023/07/14 17:18:12 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:18:12 ag(ag_cluster)[905871]: INFO: notify: 2023/07/14 17:18:12 Sequence number is 17F:F (1644972474383) >Jul 14 17:18:12 ag(ag_cluster)[905871]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:18:17 ag(ag_cluster)[905871]: INFO: ag_cluster notify : 0 >Jul 14 17:18:17 ag(ag_cluster)[905975]: INFO: mssql_validate >Jul 14 17:18:17 ag(ag_cluster)[905975]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:18:17 ag(ag_cluster)[905975]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:18:17 ag(ag_cluster)[905975]: INFO: Resource agent invoked with: notify >Jul 14 17:18:17 ag(ag_cluster)[905975]: INFO: mssql_notify pre-start >Jul 14 17:18:17 ag(ag_cluster)[905975]: INFO: notify: 2023/07/14 17:18:17 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:18:17 ag(ag_cluster)[905975]: INFO: notify: 2023/07/14 17:18:17 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:18:17 ag(ag_cluster)[905975]: INFO: notify: 2023/07/14 17:18:17 Replica is RESOLVING (0) >Jul 14 17:18:17 ag(ag_cluster)[905975]: INFO: notify: 2023/07/14 17:18:17 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:18:17 ag(ag_cluster)[905975]: INFO: notify: 2023/07/14 17:18:17 Sequence number is 17F:F (1644972474383) >Jul 14 17:18:17 ag(ag_cluster)[905975]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:18:22 ag(ag_cluster)[905975]: INFO: ag_cluster notify : 0 >Jul 14 17:18:28 ag(ag_cluster)[906075]: INFO: mssql_validate >Jul 14 17:18:28 ag(ag_cluster)[906075]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:18:28 ag(ag_cluster)[906075]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:18:28 ag(ag_cluster)[906075]: INFO: Resource agent invoked with: notify >Jul 14 17:18:28 ag(ag_cluster)[906075]: INFO: mssql_notify post-start >Jul 14 17:18:28 ag(ag_cluster)[906075]: INFO: ag_cluster notify : 0 >Jul 14 17:18:28 ag(ag_cluster)[906121]: INFO: mssql_validate >Jul 14 17:18:28 ag(ag_cluster)[906121]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:18:28 ag(ag_cluster)[906121]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:18:28 ag(ag_cluster)[906121]: INFO: Resource agent invoked with: notify >Jul 14 17:18:28 ag(ag_cluster)[906121]: INFO: mssql_notify pre-start >Jul 14 17:18:28 ag(ag_cluster)[906121]: INFO: notify: 2023/07/14 17:18:28 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:18:28 ag(ag_cluster)[906121]: INFO: notify: 2023/07/14 17:18:28 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:18:28 ag(ag_cluster)[906121]: INFO: notify: 2023/07/14 17:18:28 Replica is RESOLVING (0) >Jul 14 17:18:28 ag(ag_cluster)[906121]: INFO: notify: 2023/07/14 17:18:28 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:18:28 ag(ag_cluster)[906121]: INFO: notify: 2023/07/14 17:18:28 Sequence number is 17F:F (1644972474383) >Jul 14 17:18:28 ag(ag_cluster)[906121]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:18:33 ag(ag_cluster)[906121]: INFO: ag_cluster notify : 0 >Jul 14 17:18:33 ag(ag_cluster)[906230]: INFO: mssql_validate >Jul 14 17:18:34 ag(ag_cluster)[906230]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:18:34 ag(ag_cluster)[906230]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:18:34 ag(ag_cluster)[906230]: INFO: Resource agent invoked with: notify >Jul 14 17:18:34 ag(ag_cluster)[906230]: INFO: mssql_notify pre-start >Jul 14 17:18:34 ag(ag_cluster)[906230]: INFO: notify: 2023/07/14 17:18:34 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:18:34 ag(ag_cluster)[906230]: INFO: notify: 2023/07/14 17:18:34 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:18:34 ag(ag_cluster)[906230]: INFO: notify: 2023/07/14 17:18:34 Replica is RESOLVING (0) >Jul 14 17:18:34 ag(ag_cluster)[906230]: INFO: notify: 2023/07/14 17:18:34 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:18:34 ag(ag_cluster)[906230]: INFO: notify: 2023/07/14 17:18:34 Sequence number is 17F:F (1644972474383) >Jul 14 17:18:34 ag(ag_cluster)[906230]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:18:39 ag(ag_cluster)[906230]: INFO: ag_cluster notify : 0 >Jul 14 17:18:39 ag(ag_cluster)[906330]: INFO: mssql_validate >Jul 14 17:18:39 ag(ag_cluster)[906330]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:18:39 ag(ag_cluster)[906330]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:18:39 ag(ag_cluster)[906330]: INFO: Resource agent invoked with: notify >Jul 14 17:18:39 ag(ag_cluster)[906330]: INFO: mssql_notify post-start >Jul 14 17:18:39 ag(ag_cluster)[906330]: INFO: ag_cluster notify : 0 >Jul 14 17:18:39 ag(ag_cluster)[906376]: INFO: mssql_validate >Jul 14 17:18:39 ag(ag_cluster)[906376]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:18:39 ag(ag_cluster)[906376]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:18:39 ag(ag_cluster)[906376]: INFO: Resource agent invoked with: notify >Jul 14 17:18:39 ag(ag_cluster)[906376]: INFO: mssql_notify pre-stop >Jul 14 17:18:39 ag(ag_cluster)[906376]: INFO: ag_cluster notify : 0 >Jul 14 17:18:39 ag(ag_cluster)[906421]: INFO: mssql_validate >Jul 14 17:18:39 ag(ag_cluster)[906421]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:18:39 ag(ag_cluster)[906421]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:18:39 ag(ag_cluster)[906421]: INFO: Resource agent invoked with: notify >Jul 14 17:18:39 ag(ag_cluster)[906421]: INFO: mssql_notify post-stop >Jul 14 17:18:39 ag(ag_cluster)[906421]: INFO: notify: 2023/07/14 17:18:39 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-stop]; connection-timeout [30]; health-threshold [3]; action [post-stop] >Jul 14 17:18:39 ag(ag_cluster)[906421]: INFO: notify: 2023/07/14 17:18:39 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:18:39 ag(ag_cluster)[906421]: INFO: notify: 2023/07/14 17:18:39 Replica is RESOLVING (0) >Jul 14 17:18:39 ag(ag_cluster)[906421]: INFO: ag_cluster notify : 0 >Jul 14 17:18:39 ag(ag_cluster)[906495]: INFO: mssql_validate >Jul 14 17:18:39 ag(ag_cluster)[906495]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:18:39 ag(ag_cluster)[906495]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:18:39 ag(ag_cluster)[906495]: INFO: Resource agent invoked with: notify >Jul 14 17:18:39 ag(ag_cluster)[906495]: INFO: mssql_notify pre-promote >Jul 14 17:18:39 ag(ag_cluster)[906495]: INFO: notify: 2023/07/14 17:18:39 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-promote]; connection-timeout [30]; health-threshold [3]; action [pre-promote] >Jul 14 17:18:39 ag(ag_cluster)[906495]: INFO: notify: 2023/07/14 17:18:39 From RetryExecute - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:18:39 ag(ag_cluster)[906495]: INFO: notify: 2023/07/14 17:18:39 Connected to the instance at localhost:1433 >Jul 14 17:18:44 ag(ag_cluster)[906495]: INFO: notify: 2023/07/14 17:18:44 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:18:44 ag(ag_cluster)[906495]: INFO: notify: 2023/07/14 17:18:44 Sequence number is 17F:F (1644972474383) >Jul 14 17:18:44 ag(ag_cluster)[906495]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:18:50 ag(ag_cluster)[906495]: INFO: ag_cluster notify : 0 >Jul 14 17:18:50 ag(ag_cluster)[906598]: INFO: mssql_validate >Jul 14 17:18:50 ag(ag_cluster)[906598]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:18:50 ag(ag_cluster)[906598]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:18:50 ag(ag_cluster)[906598]: INFO: Resource agent invoked with: notify >Jul 14 17:18:50 ag(ag_cluster)[906598]: INFO: mssql_notify pre-promote >Jul 14 17:18:50 ag(ag_cluster)[906598]: INFO: notify: 2023/07/14 17:18:50 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-promote]; connection-timeout [30]; health-threshold [3]; action [pre-promote] >Jul 14 17:18:50 ag(ag_cluster)[906598]: INFO: notify: 2023/07/14 17:18:50 From RetryExecute - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:18:50 ag(ag_cluster)[906598]: INFO: notify: 2023/07/14 17:18:50 Connected to the instance at localhost:1433 >Jul 14 17:18:55 ag(ag_cluster)[906598]: INFO: notify: 2023/07/14 17:18:55 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:18:55 ag(ag_cluster)[906598]: INFO: notify: 2023/07/14 17:18:55 Sequence number is 17F:F (1644972474383) >Jul 14 17:18:55 ag(ag_cluster)[906598]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:19:00 ag(ag_cluster)[906598]: INFO: ag_cluster notify : 0 >Jul 14 17:19:06 ag(ag_cluster)[906706]: INFO: mssql_validate >Jul 14 17:19:06 ag(ag_cluster)[906706]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:19:06 ag(ag_cluster)[906706]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:19:06 ag(ag_cluster)[906706]: INFO: Resource agent invoked with: notify >Jul 14 17:19:06 ag(ag_cluster)[906706]: INFO: mssql_notify post-promote >Jul 14 17:19:06 ag(ag_cluster)[906706]: INFO: notify: 2023/07/14 17:19:06 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-promote]; connection-timeout [30]; health-threshold [3]; action [post-promote] >Jul 14 17:19:06 ag(ag_cluster)[906706]: INFO: notify: 2023/07/14 17:19:06 [DEBUG] AG Helper PostPromote Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:19:06 ag(ag_cluster)[906706]: INFO: notify: 2023/07/14 17:19:06 Replica is RESOLVING (0) >Jul 14 17:19:06 ag(ag_cluster)[906706]: INFO: ag_cluster notify : 0 >Jul 14 17:19:06 ag(ag_cluster)[906782]: INFO: mssql_validate >Jul 14 17:19:06 ag(ag_cluster)[906782]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:19:06 ag(ag_cluster)[906782]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:19:06 ag(ag_cluster)[906782]: INFO: Resource agent invoked with: monitor >Jul 14 17:19:06 ag(ag_cluster)[906782]: INFO: mssql_monitor >Jul 14 17:19:06 ag(ag_cluster)[906782]: INFO: monitor: 2023/07/14 17:19:06 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:19:06 ag(ag_cluster)[906782]: INFO: monitor: 2023/07/14 17:19:06 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:19:06 ag(ag_cluster)[906782]: INFO: monitor: 2023/07/14 17:19:06 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:19:06 ag(ag_cluster)[906782]: INFO: monitor: 2023/07/14 17:19:06 Connected to the instance at localhost:1433 >Jul 14 17:19:11 ag(ag_cluster)[906782]: INFO: monitor: 2023/07/14 17:19:11 Monitor Caller is: monitor. >Jul 14 17:19:11 ag(ag_cluster)[906782]: INFO: monitor: 2023/07/14 17:19:11 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:19:11 ag(ag_cluster)[906782]: INFO: monitor: 2023/07/14 17:19:11 Replica is RESOLVING (0) >Jul 14 17:19:11 ag(ag_cluster)[906782]: INFO: monitor: 2023/07/14 17:19:11 Instance name is server3. >Jul 14 17:19:11 ag(ag_cluster)[906782]: INFO: monitor: 2023/07/14 17:19:11 Current master is server2. >Jul 14 17:19:11 ag(ag_cluster)[906782]: INFO: monitor: 2023/07/14 17:19:11 Setting the role to Secondary. >Jul 14 17:19:11 ag(ag_cluster)[906782]: INFO: monitor: 2023/07/14 17:19:11 Setting replica to SECONDARY role... >Jul 14 17:19:11 ag(ag_cluster)[906782]: INFO: monitor: 2023/07/14 17:19:11 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:19:11 ag(ag_cluster)[906782]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:19:11 ag(ag_cluster)[906782]: INFO: lease_expiry after monitor update: >Jul 14 17:19:11 ag(ag_cluster)[906782]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:19:11 ag(ag_cluster)[906782]: INFO: ag_cluster monitor : 0 >Jul 14 17:19:11 ag(ag_cluster)[906954]: INFO: mssql_validate >Jul 14 17:19:11 ag(ag_cluster)[906954]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:19:11 ag(ag_cluster)[906954]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:19:11 ag(ag_cluster)[906954]: INFO: Resource agent invoked with: monitor >Jul 14 17:19:11 ag(ag_cluster)[906954]: INFO: mssql_monitor >Jul 14 17:19:11 ag(ag_cluster)[906954]: INFO: monitor: 2023/07/14 17:19:11 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:19:11 ag(ag_cluster)[906954]: INFO: monitor: 2023/07/14 17:19:11 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:19:11 ag(ag_cluster)[906954]: INFO: monitor: 2023/07/14 17:19:11 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:19:11 ag(ag_cluster)[906954]: INFO: monitor: 2023/07/14 17:19:11 Connected to the instance at localhost:1433 >Jul 14 17:19:16 ag(ag_cluster)[906954]: INFO: monitor: 2023/07/14 17:19:16 Monitor Caller is: monitor. >Jul 14 17:19:16 ag(ag_cluster)[906954]: INFO: monitor: 2023/07/14 17:19:16 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:19:16 ag(ag_cluster)[906954]: INFO: monitor: 2023/07/14 17:19:16 Replica is RESOLVING (0) >Jul 14 17:19:16 ag(ag_cluster)[906954]: INFO: monitor: 2023/07/14 17:19:16 Instance name is server3. >Jul 14 17:19:16 ag(ag_cluster)[906954]: INFO: monitor: 2023/07/14 17:19:16 Current master is server2. >Jul 14 17:19:16 ag(ag_cluster)[906954]: INFO: monitor: 2023/07/14 17:19:16 Setting the role to Secondary. >Jul 14 17:19:16 ag(ag_cluster)[906954]: INFO: monitor: 2023/07/14 17:19:16 Setting replica to SECONDARY role... >Jul 14 17:19:16 ag(ag_cluster)[906954]: INFO: monitor: 2023/07/14 17:19:16 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:19:16 ag(ag_cluster)[906954]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:19:16 ag(ag_cluster)[906954]: INFO: lease_expiry after monitor update: >Jul 14 17:19:16 ag(ag_cluster)[906954]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:19:17 ag(ag_cluster)[906954]: INFO: ag_cluster monitor : 0 >Jul 14 17:19:23 ag(ag_cluster)[907132]: INFO: mssql_validate >Jul 14 17:19:23 ag(ag_cluster)[907132]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:19:23 ag(ag_cluster)[907132]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:19:23 ag(ag_cluster)[907132]: INFO: Resource agent invoked with: monitor >Jul 14 17:19:23 ag(ag_cluster)[907132]: INFO: mssql_monitor >Jul 14 17:19:23 ag(ag_cluster)[907132]: INFO: monitor: 2023/07/14 17:19:23 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:19:23 ag(ag_cluster)[907132]: INFO: monitor: 2023/07/14 17:19:23 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:19:23 ag(ag_cluster)[907132]: INFO: monitor: 2023/07/14 17:19:23 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:19:23 ag(ag_cluster)[907132]: INFO: monitor: 2023/07/14 17:19:23 Connected to the instance at localhost:1433 >Jul 14 17:19:28 ag(ag_cluster)[907132]: INFO: monitor: 2023/07/14 17:19:28 Monitor Caller is: monitor. >Jul 14 17:19:28 ag(ag_cluster)[907132]: INFO: monitor: 2023/07/14 17:19:28 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:19:28 ag(ag_cluster)[907132]: INFO: monitor: 2023/07/14 17:19:28 Replica is RESOLVING (0) >Jul 14 17:19:28 ag(ag_cluster)[907132]: INFO: monitor: 2023/07/14 17:19:28 Instance name is server3. >Jul 14 17:19:28 ag(ag_cluster)[907132]: INFO: monitor: 2023/07/14 17:19:28 Current master is server2. >Jul 14 17:19:28 ag(ag_cluster)[907132]: INFO: monitor: 2023/07/14 17:19:28 Setting the role to Secondary. >Jul 14 17:19:28 ag(ag_cluster)[907132]: INFO: monitor: 2023/07/14 17:19:28 Setting replica to SECONDARY role... >Jul 14 17:19:28 ag(ag_cluster)[907132]: INFO: monitor: 2023/07/14 17:19:28 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:19:28 ag(ag_cluster)[907132]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:19:28 ag(ag_cluster)[907132]: INFO: lease_expiry after monitor update: >Jul 14 17:19:28 ag(ag_cluster)[907132]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:19:29 ag(ag_cluster)[907132]: INFO: ag_cluster monitor : 0 >Jul 14 17:19:29 ag(ag_cluster)[907303]: INFO: mssql_validate >Jul 14 17:19:29 ag(ag_cluster)[907303]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:19:29 ag(ag_cluster)[907303]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:19:29 ag(ag_cluster)[907303]: INFO: Resource agent invoked with: monitor >Jul 14 17:19:29 ag(ag_cluster)[907303]: INFO: mssql_monitor >Jul 14 17:19:29 ag(ag_cluster)[907303]: INFO: monitor: 2023/07/14 17:19:29 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:19:29 ag(ag_cluster)[907303]: INFO: monitor: 2023/07/14 17:19:29 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:19:29 ag(ag_cluster)[907303]: INFO: monitor: 2023/07/14 17:19:29 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:19:29 ag(ag_cluster)[907303]: INFO: monitor: 2023/07/14 17:19:29 Connected to the instance at localhost:1433 >Jul 14 17:19:34 ag(ag_cluster)[907303]: INFO: monitor: 2023/07/14 17:19:34 Monitor Caller is: monitor. >Jul 14 17:19:34 ag(ag_cluster)[907303]: INFO: monitor: 2023/07/14 17:19:34 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:19:34 ag(ag_cluster)[907303]: INFO: monitor: 2023/07/14 17:19:34 Replica is RESOLVING (0) >Jul 14 17:19:34 ag(ag_cluster)[907303]: INFO: monitor: 2023/07/14 17:19:34 Instance name is server3. >Jul 14 17:19:34 ag(ag_cluster)[907303]: INFO: monitor: 2023/07/14 17:19:34 Current master is server2. >Jul 14 17:19:34 ag(ag_cluster)[907303]: INFO: monitor: 2023/07/14 17:19:34 Setting the role to Secondary. >Jul 14 17:19:34 ag(ag_cluster)[907303]: INFO: monitor: 2023/07/14 17:19:34 Setting replica to SECONDARY role... >Jul 14 17:19:34 ag(ag_cluster)[907303]: INFO: monitor: 2023/07/14 17:19:34 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:19:34 ag(ag_cluster)[907303]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:19:34 ag(ag_cluster)[907303]: INFO: lease_expiry after monitor update: >Jul 14 17:19:34 ag(ag_cluster)[907303]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:19:34 ag(ag_cluster)[907303]: INFO: ag_cluster monitor : 0 >Jul 14 17:19:40 ag(ag_cluster)[907484]: INFO: mssql_validate >Jul 14 17:19:40 ag(ag_cluster)[907484]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:19:40 ag(ag_cluster)[907484]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:19:40 ag(ag_cluster)[907484]: INFO: Resource agent invoked with: notify >Jul 14 17:19:40 ag(ag_cluster)[907484]: INFO: mssql_notify pre-start >Jul 14 17:19:40 ag(ag_cluster)[907484]: INFO: notify: 2023/07/14 17:19:40 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:19:40 ag(ag_cluster)[907484]: INFO: notify: 2023/07/14 17:19:40 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:19:40 ag(ag_cluster)[907484]: INFO: notify: 2023/07/14 17:19:40 Replica is RESOLVING (0) >Jul 14 17:19:40 ag(ag_cluster)[907484]: INFO: notify: 2023/07/14 17:19:40 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:19:40 ag(ag_cluster)[907484]: INFO: notify: 2023/07/14 17:19:40 Sequence number is 17F:F (1644972474383) >Jul 14 17:19:40 ag(ag_cluster)[907484]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:19:45 ag(ag_cluster)[907484]: INFO: ag_cluster notify : 0 >Jul 14 17:19:45 ag(ag_cluster)[907583]: INFO: mssql_validate >Jul 14 17:19:45 ag(ag_cluster)[907583]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:19:45 ag(ag_cluster)[907583]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:19:45 ag(ag_cluster)[907583]: INFO: Resource agent invoked with: monitor >Jul 14 17:19:45 ag(ag_cluster)[907583]: INFO: mssql_monitor >Jul 14 17:19:45 ag(ag_cluster)[907583]: INFO: monitor: 2023/07/14 17:19:45 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:19:45 ag(ag_cluster)[907583]: INFO: monitor: 2023/07/14 17:19:45 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:19:45 ag(ag_cluster)[907583]: INFO: monitor: 2023/07/14 17:19:45 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:19:45 ag(ag_cluster)[907583]: INFO: monitor: 2023/07/14 17:19:45 Connected to the instance at localhost:1433 >Jul 14 17:19:50 ag(ag_cluster)[907583]: INFO: monitor: 2023/07/14 17:19:50 Monitor Caller is: monitor. >Jul 14 17:19:50 ag(ag_cluster)[907583]: INFO: monitor: 2023/07/14 17:19:50 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:19:50 ag(ag_cluster)[907583]: INFO: monitor: 2023/07/14 17:19:50 Replica is RESOLVING (0) >Jul 14 17:19:50 ag(ag_cluster)[907583]: INFO: monitor: 2023/07/14 17:19:50 Instance name is server3. >Jul 14 17:19:50 ag(ag_cluster)[907583]: INFO: monitor: 2023/07/14 17:19:50 Current master is server2. >Jul 14 17:19:50 ag(ag_cluster)[907583]: INFO: monitor: 2023/07/14 17:19:50 Setting the role to Secondary. >Jul 14 17:19:50 ag(ag_cluster)[907583]: INFO: monitor: 2023/07/14 17:19:50 Setting replica to SECONDARY role... >Jul 14 17:19:50 ag(ag_cluster)[907583]: INFO: monitor: 2023/07/14 17:19:50 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:19:50 ag(ag_cluster)[907583]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:19:50 ag(ag_cluster)[907583]: INFO: lease_expiry after monitor update: >Jul 14 17:19:50 ag(ag_cluster)[907583]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:19:51 ag(ag_cluster)[907583]: INFO: ag_cluster monitor : 0 >Jul 14 17:19:51 ag(ag_cluster)[907755]: INFO: mssql_validate >Jul 14 17:19:51 ag(ag_cluster)[907755]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:19:51 ag(ag_cluster)[907755]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:19:51 ag(ag_cluster)[907755]: INFO: Resource agent invoked with: monitor >Jul 14 17:19:51 ag(ag_cluster)[907755]: INFO: mssql_monitor >Jul 14 17:19:51 ag(ag_cluster)[907755]: INFO: monitor: 2023/07/14 17:19:51 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:19:51 ag(ag_cluster)[907755]: INFO: monitor: 2023/07/14 17:19:51 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:19:51 ag(ag_cluster)[907755]: INFO: monitor: 2023/07/14 17:19:51 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:19:51 ag(ag_cluster)[907755]: INFO: monitor: 2023/07/14 17:19:51 Connected to the instance at localhost:1433 >Jul 14 17:19:56 ag(ag_cluster)[907755]: INFO: monitor: 2023/07/14 17:19:56 Monitor Caller is: monitor. >Jul 14 17:19:56 ag(ag_cluster)[907755]: INFO: monitor: 2023/07/14 17:19:56 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:19:56 ag(ag_cluster)[907755]: INFO: monitor: 2023/07/14 17:19:56 Replica is RESOLVING (0) >Jul 14 17:19:56 ag(ag_cluster)[907755]: INFO: monitor: 2023/07/14 17:19:56 Instance name is server3. >Jul 14 17:19:56 ag(ag_cluster)[907755]: INFO: monitor: 2023/07/14 17:19:56 Current master is server2. >Jul 14 17:19:56 ag(ag_cluster)[907755]: INFO: monitor: 2023/07/14 17:19:56 Setting the role to Secondary. >Jul 14 17:19:56 ag(ag_cluster)[907755]: INFO: monitor: 2023/07/14 17:19:56 Setting replica to SECONDARY role... >Jul 14 17:19:56 ag(ag_cluster)[907755]: INFO: monitor: 2023/07/14 17:19:56 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:19:56 ag(ag_cluster)[907755]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:19:56 ag(ag_cluster)[907755]: INFO: lease_expiry after monitor update: >Jul 14 17:19:56 ag(ag_cluster)[907755]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:19:56 ag(ag_cluster)[907755]: INFO: ag_cluster monitor : 0 >Jul 14 17:19:56 ag(ag_cluster)[907927]: INFO: mssql_validate >Jul 14 17:19:56 ag(ag_cluster)[907927]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:19:56 ag(ag_cluster)[907927]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:19:56 ag(ag_cluster)[907927]: INFO: Resource agent invoked with: notify >Jul 14 17:19:56 ag(ag_cluster)[907927]: INFO: mssql_notify pre-start >Jul 14 17:19:56 ag(ag_cluster)[907927]: INFO: notify: 2023/07/14 17:19:56 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:19:56 ag(ag_cluster)[907927]: INFO: notify: 2023/07/14 17:19:56 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:19:56 ag(ag_cluster)[907927]: INFO: notify: 2023/07/14 17:19:56 Replica is RESOLVING (0) >Jul 14 17:19:56 ag(ag_cluster)[907927]: INFO: notify: 2023/07/14 17:19:56 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:19:56 ag(ag_cluster)[907927]: INFO: notify: 2023/07/14 17:19:56 Sequence number is 17F:F (1644972474383) >Jul 14 17:19:56 ag(ag_cluster)[907927]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:20:01 ag(ag_cluster)[907927]: INFO: ag_cluster notify : 0 >Jul 14 17:20:02 ag(ag_cluster)[908027]: INFO: mssql_validate >Jul 14 17:20:02 ag(ag_cluster)[908027]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:20:02 ag(ag_cluster)[908027]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:20:02 ag(ag_cluster)[908027]: INFO: Resource agent invoked with: notify >Jul 14 17:20:02 ag(ag_cluster)[908027]: INFO: mssql_notify post-start >Jul 14 17:20:02 ag(ag_cluster)[908027]: INFO: ag_cluster notify : 0 >Jul 14 17:20:02 ag(ag_cluster)[908073]: INFO: mssql_validate >Jul 14 17:20:02 ag(ag_cluster)[908073]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:20:02 ag(ag_cluster)[908073]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:20:02 ag(ag_cluster)[908073]: INFO: Resource agent invoked with: notify >Jul 14 17:20:02 ag(ag_cluster)[908073]: INFO: mssql_notify pre-stop >Jul 14 17:20:02 ag(ag_cluster)[908073]: INFO: ag_cluster notify : 0 >Jul 14 17:20:02 ag(ag_cluster)[908118]: INFO: mssql_validate >Jul 14 17:20:02 ag(ag_cluster)[908118]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:20:02 ag(ag_cluster)[908118]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:20:02 ag(ag_cluster)[908118]: INFO: Resource agent invoked with: notify >Jul 14 17:20:02 ag(ag_cluster)[908118]: INFO: mssql_notify post-stop >Jul 14 17:20:02 ag(ag_cluster)[908118]: INFO: notify: 2023/07/14 17:20:02 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-stop]; connection-timeout [30]; health-threshold [3]; action [post-stop] >Jul 14 17:20:02 ag(ag_cluster)[908118]: INFO: notify: 2023/07/14 17:20:02 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:20:02 ag(ag_cluster)[908118]: INFO: notify: 2023/07/14 17:20:02 Replica is RESOLVING (0) >Jul 14 17:20:02 ag(ag_cluster)[908118]: INFO: ag_cluster notify : 0 >Jul 14 17:20:03 ag(ag_cluster)[908193]: INFO: mssql_validate >Jul 14 17:20:03 ag(ag_cluster)[908193]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:20:03 ag(ag_cluster)[908193]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:20:03 ag(ag_cluster)[908193]: INFO: Resource agent invoked with: monitor >Jul 14 17:20:03 ag(ag_cluster)[908193]: INFO: mssql_monitor >Jul 14 17:20:03 ag(ag_cluster)[908193]: INFO: monitor: 2023/07/14 17:20:03 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:20:03 ag(ag_cluster)[908193]: INFO: monitor: 2023/07/14 17:20:03 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:20:03 ag(ag_cluster)[908193]: INFO: monitor: 2023/07/14 17:20:03 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:20:03 ag(ag_cluster)[908193]: INFO: monitor: 2023/07/14 17:20:03 Connected to the instance at localhost:1433 >Jul 14 17:20:08 ag(ag_cluster)[908193]: INFO: monitor: 2023/07/14 17:20:08 Monitor Caller is: monitor. >Jul 14 17:20:08 ag(ag_cluster)[908193]: INFO: monitor: 2023/07/14 17:20:08 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:20:08 ag(ag_cluster)[908193]: INFO: monitor: 2023/07/14 17:20:08 Replica is RESOLVING (0) >Jul 14 17:20:08 ag(ag_cluster)[908193]: INFO: monitor: 2023/07/14 17:20:08 Instance name is server3. >Jul 14 17:20:08 ag(ag_cluster)[908193]: INFO: monitor: 2023/07/14 17:20:08 Current master is server2. >Jul 14 17:20:08 ag(ag_cluster)[908193]: INFO: monitor: 2023/07/14 17:20:08 Setting the role to Secondary. >Jul 14 17:20:08 ag(ag_cluster)[908193]: INFO: monitor: 2023/07/14 17:20:08 Setting replica to SECONDARY role... >Jul 14 17:20:08 ag(ag_cluster)[908193]: INFO: monitor: 2023/07/14 17:20:08 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:20:08 ag(ag_cluster)[908193]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:20:08 ag(ag_cluster)[908193]: INFO: lease_expiry after monitor update: >Jul 14 17:20:08 ag(ag_cluster)[908193]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:20:08 ag(ag_cluster)[908193]: INFO: ag_cluster monitor : 0 >Jul 14 17:20:08 ag(ag_cluster)[908366]: INFO: mssql_validate >Jul 14 17:20:08 ag(ag_cluster)[908366]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:20:08 ag(ag_cluster)[908366]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:20:08 ag(ag_cluster)[908366]: INFO: Resource agent invoked with: monitor >Jul 14 17:20:08 ag(ag_cluster)[908366]: INFO: mssql_monitor >Jul 14 17:20:08 ag(ag_cluster)[908366]: INFO: monitor: 2023/07/14 17:20:08 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:20:08 ag(ag_cluster)[908366]: INFO: monitor: 2023/07/14 17:20:08 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:20:08 ag(ag_cluster)[908366]: INFO: monitor: 2023/07/14 17:20:08 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:20:08 ag(ag_cluster)[908366]: INFO: monitor: 2023/07/14 17:20:08 Connected to the instance at localhost:1433 >Jul 14 17:20:13 ag(ag_cluster)[908366]: INFO: monitor: 2023/07/14 17:20:13 Monitor Caller is: monitor. >Jul 14 17:20:13 ag(ag_cluster)[908366]: INFO: monitor: 2023/07/14 17:20:13 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:20:13 ag(ag_cluster)[908366]: INFO: monitor: 2023/07/14 17:20:13 Replica is RESOLVING (0) >Jul 14 17:20:13 ag(ag_cluster)[908366]: INFO: monitor: 2023/07/14 17:20:13 Instance name is server3. >Jul 14 17:20:13 ag(ag_cluster)[908366]: INFO: monitor: 2023/07/14 17:20:13 Current master is server2. >Jul 14 17:20:13 ag(ag_cluster)[908366]: INFO: monitor: 2023/07/14 17:20:13 Setting the role to Secondary. >Jul 14 17:20:13 ag(ag_cluster)[908366]: INFO: monitor: 2023/07/14 17:20:13 Setting replica to SECONDARY role... >Jul 14 17:20:13 ag(ag_cluster)[908366]: INFO: monitor: 2023/07/14 17:20:13 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:20:13 ag(ag_cluster)[908366]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:20:13 ag(ag_cluster)[908366]: INFO: lease_expiry after monitor update: >Jul 14 17:20:13 ag(ag_cluster)[908366]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:20:14 ag(ag_cluster)[908366]: INFO: ag_cluster monitor : 0 >Jul 14 17:20:20 ag(ag_cluster)[908544]: INFO: mssql_validate >Jul 14 17:20:20 ag(ag_cluster)[908544]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:20:20 ag(ag_cluster)[908544]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:20:20 ag(ag_cluster)[908544]: INFO: Resource agent invoked with: monitor >Jul 14 17:20:20 ag(ag_cluster)[908544]: INFO: mssql_monitor >Jul 14 17:20:20 ag(ag_cluster)[908544]: INFO: monitor: 2023/07/14 17:20:20 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:20:20 ag(ag_cluster)[908544]: INFO: monitor: 2023/07/14 17:20:20 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:20:20 ag(ag_cluster)[908544]: INFO: monitor: 2023/07/14 17:20:20 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:20:20 ag(ag_cluster)[908544]: INFO: monitor: 2023/07/14 17:20:20 Connected to the instance at localhost:1433 >Jul 14 17:20:25 ag(ag_cluster)[908544]: INFO: monitor: 2023/07/14 17:20:25 Monitor Caller is: monitor. >Jul 14 17:20:25 ag(ag_cluster)[908544]: INFO: monitor: 2023/07/14 17:20:25 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:20:25 ag(ag_cluster)[908544]: INFO: monitor: 2023/07/14 17:20:25 Replica is RESOLVING (0) >Jul 14 17:20:25 ag(ag_cluster)[908544]: INFO: monitor: 2023/07/14 17:20:25 Instance name is server3. >Jul 14 17:20:25 ag(ag_cluster)[908544]: INFO: monitor: 2023/07/14 17:20:25 Current master is server2. >Jul 14 17:20:25 ag(ag_cluster)[908544]: INFO: monitor: 2023/07/14 17:20:25 Setting the role to Secondary. >Jul 14 17:20:25 ag(ag_cluster)[908544]: INFO: monitor: 2023/07/14 17:20:25 Setting replica to SECONDARY role... >Jul 14 17:20:25 ag(ag_cluster)[908544]: INFO: monitor: 2023/07/14 17:20:25 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:20:25 ag(ag_cluster)[908544]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:20:26 ag(ag_cluster)[908544]: INFO: lease_expiry after monitor update: >Jul 14 17:20:26 ag(ag_cluster)[908544]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:20:26 ag(ag_cluster)[908544]: INFO: ag_cluster monitor : 0 >Jul 14 17:20:26 ag(ag_cluster)[908716]: INFO: mssql_validate >Jul 14 17:20:26 ag(ag_cluster)[908716]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:20:26 ag(ag_cluster)[908716]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:20:26 ag(ag_cluster)[908716]: INFO: Resource agent invoked with: monitor >Jul 14 17:20:26 ag(ag_cluster)[908716]: INFO: mssql_monitor >Jul 14 17:20:26 ag(ag_cluster)[908716]: INFO: monitor: 2023/07/14 17:20:26 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:20:26 ag(ag_cluster)[908716]: INFO: monitor: 2023/07/14 17:20:26 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:20:26 ag(ag_cluster)[908716]: INFO: monitor: 2023/07/14 17:20:26 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:20:26 ag(ag_cluster)[908716]: INFO: monitor: 2023/07/14 17:20:26 Connected to the instance at localhost:1433 >Jul 14 17:20:31 ag(ag_cluster)[908716]: INFO: monitor: 2023/07/14 17:20:31 Monitor Caller is: monitor. >Jul 14 17:20:31 ag(ag_cluster)[908716]: INFO: monitor: 2023/07/14 17:20:31 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:20:31 ag(ag_cluster)[908716]: INFO: monitor: 2023/07/14 17:20:31 Replica is RESOLVING (0) >Jul 14 17:20:31 ag(ag_cluster)[908716]: INFO: monitor: 2023/07/14 17:20:31 Instance name is server3. >Jul 14 17:20:31 ag(ag_cluster)[908716]: INFO: monitor: 2023/07/14 17:20:31 Current master is server2. >Jul 14 17:20:31 ag(ag_cluster)[908716]: INFO: monitor: 2023/07/14 17:20:31 Setting the role to Secondary. >Jul 14 17:20:31 ag(ag_cluster)[908716]: INFO: monitor: 2023/07/14 17:20:31 Setting replica to SECONDARY role... >Jul 14 17:20:31 ag(ag_cluster)[908716]: INFO: monitor: 2023/07/14 17:20:31 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:20:31 ag(ag_cluster)[908716]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:20:31 ag(ag_cluster)[908716]: INFO: lease_expiry after monitor update: >Jul 14 17:20:31 ag(ag_cluster)[908716]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:20:31 ag(ag_cluster)[908716]: INFO: ag_cluster monitor : 0 >Jul 14 17:20:34 ag(ag_cluster)[908897]: INFO: mssql_validate >Jul 14 17:20:34 ag(ag_cluster)[908897]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:20:34 ag(ag_cluster)[908897]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:20:34 ag(ag_cluster)[908897]: INFO: Resource agent invoked with: notify >Jul 14 17:20:34 ag(ag_cluster)[908897]: INFO: mssql_notify pre-demote >Jul 14 17:20:34 ag(ag_cluster)[908897]: INFO: ag_cluster notify : 0 >Jul 14 17:20:34 ag(ag_cluster)[908942]: INFO: mssql_validate >Jul 14 17:20:34 ag(ag_cluster)[908942]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:20:34 ag(ag_cluster)[908942]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:20:34 ag(ag_cluster)[908942]: INFO: Resource agent invoked with: notify >Jul 14 17:20:34 ag(ag_cluster)[908942]: INFO: mssql_notify post-demote >Jul 14 17:20:34 ag(ag_cluster)[908942]: INFO: ag_cluster notify : 0 >Jul 14 17:20:34 ag(ag_cluster)[908987]: INFO: mssql_validate >Jul 14 17:20:34 ag(ag_cluster)[908987]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:20:34 ag(ag_cluster)[908987]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:20:34 ag(ag_cluster)[908987]: INFO: Resource agent invoked with: notify >Jul 14 17:20:34 ag(ag_cluster)[908987]: INFO: mssql_notify pre-stop >Jul 14 17:20:34 ag(ag_cluster)[908987]: INFO: ag_cluster notify : 0 >Jul 14 17:20:34 ag(ag_cluster)[909032]: INFO: mssql_validate >Jul 14 17:20:34 ag(ag_cluster)[909032]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:20:34 ag(ag_cluster)[909032]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:20:34 ag(ag_cluster)[909032]: INFO: Resource agent invoked with: notify >Jul 14 17:20:35 ag(ag_cluster)[909032]: INFO: mssql_notify post-stop >Jul 14 17:20:35 ag(ag_cluster)[909032]: INFO: notify: 2023/07/14 17:20:35 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-stop]; connection-timeout [30]; health-threshold [3]; action [post-stop] >Jul 14 17:20:35 ag(ag_cluster)[909032]: INFO: notify: 2023/07/14 17:20:35 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:20:35 ag(ag_cluster)[909032]: INFO: notify: 2023/07/14 17:20:35 Replica is RESOLVING (0) >Jul 14 17:20:35 ag(ag_cluster)[909032]: INFO: ag_cluster notify : 0 >Jul 14 17:20:35 ag(ag_cluster)[909106]: INFO: mssql_validate >Jul 14 17:20:35 ag(ag_cluster)[909106]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:20:35 ag(ag_cluster)[909106]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:20:35 ag(ag_cluster)[909106]: INFO: Resource agent invoked with: notify >Jul 14 17:20:35 ag(ag_cluster)[909106]: INFO: mssql_notify pre-start >Jul 14 17:20:35 ag(ag_cluster)[909106]: INFO: notify: 2023/07/14 17:20:35 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:20:35 ag(ag_cluster)[909106]: INFO: notify: 2023/07/14 17:20:35 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:20:35 ag(ag_cluster)[909106]: INFO: notify: 2023/07/14 17:20:35 Replica is RESOLVING (0) >Jul 14 17:20:35 ag(ag_cluster)[909106]: INFO: notify: 2023/07/14 17:20:35 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:20:35 ag(ag_cluster)[909106]: INFO: notify: 2023/07/14 17:20:35 Sequence number is 17F:F (1644972474383) >Jul 14 17:20:35 ag(ag_cluster)[909106]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:20:40 ag(ag_cluster)[909106]: INFO: ag_cluster notify : 0 >Jul 14 17:20:40 ag(ag_cluster)[909206]: INFO: mssql_validate >Jul 14 17:20:40 ag(ag_cluster)[909206]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:20:40 ag(ag_cluster)[909206]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:20:40 ag(ag_cluster)[909206]: INFO: Resource agent invoked with: monitor >Jul 14 17:20:40 ag(ag_cluster)[909206]: INFO: mssql_monitor >Jul 14 17:20:40 ag(ag_cluster)[909206]: INFO: monitor: 2023/07/14 17:20:40 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:20:40 ag(ag_cluster)[909206]: INFO: monitor: 2023/07/14 17:20:40 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master []; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:20:40 ag(ag_cluster)[909206]: INFO: monitor: 2023/07/14 17:20:40 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:20:40 ag(ag_cluster)[909206]: INFO: monitor: 2023/07/14 17:20:40 Connected to the instance at localhost:1433 >Jul 14 17:20:45 ag(ag_cluster)[909206]: INFO: monitor: 2023/07/14 17:20:45 Monitor Caller is: monitor. >Jul 14 17:20:45 ag(ag_cluster)[909206]: INFO: monitor: 2023/07/14 17:20:45 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:20:45 ag(ag_cluster)[909206]: INFO: monitor: 2023/07/14 17:20:45 Replica is RESOLVING (0) >Jul 14 17:20:45 ag(ag_cluster)[909206]: INFO: monitor: 2023/07/14 17:20:45 Instance name is server3. >Jul 14 17:20:45 ag(ag_cluster)[909206]: INFO: monitor: 2023/07/14 17:20:45 Current master is . >Jul 14 17:20:45 ag(ag_cluster)[909206]: INFO: monitor: PROMOTION_SCORE: 10 >Jul 14 17:20:45 ag(ag_cluster)[909206]: INFO: lease_expiry after monitor update: >Jul 14 17:20:45 ag(ag_cluster)[909206]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:20:45 ag(ag_cluster)[909206]: INFO: ag_cluster monitor : 0 >Jul 14 17:20:45 ag(ag_cluster)[909360]: INFO: mssql_validate >Jul 14 17:20:45 ag(ag_cluster)[909360]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:20:45 ag(ag_cluster)[909360]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:20:45 ag(ag_cluster)[909360]: INFO: Resource agent invoked with: notify >Jul 14 17:20:45 ag(ag_cluster)[909360]: INFO: mssql_notify pre-start >Jul 14 17:20:45 ag(ag_cluster)[909360]: INFO: notify: 2023/07/14 17:20:45 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:20:45 ag(ag_cluster)[909360]: INFO: notify: 2023/07/14 17:20:45 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:20:45 ag(ag_cluster)[909360]: INFO: notify: 2023/07/14 17:20:45 Replica is RESOLVING (0) >Jul 14 17:20:45 ag(ag_cluster)[909360]: INFO: notify: 2023/07/14 17:20:45 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:20:45 ag(ag_cluster)[909360]: INFO: notify: 2023/07/14 17:20:45 Sequence number is 17F:F (1644972474383) >Jul 14 17:20:45 ag(ag_cluster)[909360]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:20:51 ag(ag_cluster)[909360]: INFO: ag_cluster notify : 0 >Jul 14 17:20:51 ag(ag_cluster)[909459]: INFO: mssql_validate >Jul 14 17:20:51 ag(ag_cluster)[909459]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:20:51 ag(ag_cluster)[909476]: INFO: mssql_validate >Jul 14 17:20:51 ag(ag_cluster)[909476]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:20:51 ag(ag_cluster)[909476]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:20:51 ag(ag_cluster)[909476]: INFO: Resource agent invoked with: notify >Jul 14 17:20:51 ag(ag_cluster)[909476]: INFO: mssql_notify pre-start >Jul 14 17:20:51 ag(ag_cluster)[909476]: INFO: notify: 2023/07/14 17:20:51 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:20:51 ag(ag_cluster)[909476]: INFO: notify: 2023/07/14 17:20:51 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:20:51 ag(ag_cluster)[909476]: INFO: notify: 2023/07/14 17:20:51 Replica is RESOLVING (0) >Jul 14 17:20:51 ag(ag_cluster)[909476]: INFO: notify: 2023/07/14 17:20:51 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:20:51 ag(ag_cluster)[909476]: INFO: notify: 2023/07/14 17:20:51 Sequence number is 17F:F (1644972474383) >Jul 14 17:20:51 ag(ag_cluster)[909476]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:20:56 ag(ag_cluster)[909476]: INFO: ag_cluster notify : 0 >Jul 14 17:21:01 ag(ag_cluster)[909577]: INFO: mssql_validate >Jul 14 17:21:01 ag(ag_cluster)[909577]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:21:01 ag(ag_cluster)[909577]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:21:01 ag(ag_cluster)[909577]: INFO: Resource agent invoked with: notify >Jul 14 17:21:01 ag(ag_cluster)[909577]: INFO: mssql_notify post-start >Jul 14 17:21:01 ag(ag_cluster)[909577]: INFO: ag_cluster notify : 0 >Jul 14 17:21:02 ag(ag_cluster)[909623]: INFO: mssql_validate >Jul 14 17:21:02 ag(ag_cluster)[909623]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:21:02 ag(ag_cluster)[909623]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:21:02 ag(ag_cluster)[909623]: INFO: Resource agent invoked with: notify >Jul 14 17:21:02 ag(ag_cluster)[909623]: INFO: mssql_notify pre-promote >Jul 14 17:21:02 ag(ag_cluster)[909623]: INFO: notify: 2023/07/14 17:21:02 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-promote]; connection-timeout [30]; health-threshold [3]; action [pre-promote] >Jul 14 17:21:02 ag(ag_cluster)[909623]: INFO: notify: 2023/07/14 17:21:02 From RetryExecute - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:21:02 ag(ag_cluster)[909623]: INFO: notify: 2023/07/14 17:21:02 Connected to the instance at localhost:1433 >Jul 14 17:21:07 ag(ag_cluster)[909623]: INFO: notify: 2023/07/14 17:21:07 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:21:07 ag(ag_cluster)[909623]: INFO: notify: 2023/07/14 17:21:07 Sequence number is 17F:F (1644972474383) >Jul 14 17:21:07 ag(ag_cluster)[909623]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:21:12 ag(ag_cluster)[909623]: INFO: ag_cluster notify : 0 >Jul 14 17:21:12 ag(ag_cluster)[909728]: INFO: mssql_validate >Jul 14 17:21:12 ag(ag_cluster)[909728]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:21:12 ag(ag_cluster)[909728]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:21:12 ag(ag_cluster)[909728]: INFO: Resource agent invoked with: notify >Jul 14 17:21:12 ag(ag_cluster)[909728]: INFO: mssql_notify pre-start >Jul 14 17:21:12 ag(ag_cluster)[909728]: INFO: notify: 2023/07/14 17:21:12 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:21:12 ag(ag_cluster)[909728]: INFO: notify: 2023/07/14 17:21:12 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:21:12 ag(ag_cluster)[909728]: INFO: notify: 2023/07/14 17:21:12 Replica is RESOLVING (0) >Jul 14 17:21:12 ag(ag_cluster)[909728]: INFO: notify: 2023/07/14 17:21:12 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:21:12 ag(ag_cluster)[909728]: INFO: notify: 2023/07/14 17:21:12 Sequence number is 17F:F (1644972474383) >Jul 14 17:21:12 ag(ag_cluster)[909728]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:21:17 ag(ag_cluster)[909728]: INFO: ag_cluster notify : 0 >Jul 14 17:21:17 ag(ag_cluster)[909833]: INFO: mssql_validate >Jul 14 17:21:17 ag(ag_cluster)[909833]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:21:17 ag(ag_cluster)[909833]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:21:17 ag(ag_cluster)[909833]: INFO: Resource agent invoked with: notify >Jul 14 17:21:17 ag(ag_cluster)[909833]: INFO: mssql_notify pre-start >Jul 14 17:21:17 ag(ag_cluster)[909833]: INFO: notify: 2023/07/14 17:21:17 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-start]; connection-timeout [30]; health-threshold [3]; action [pre-start] >Jul 14 17:21:17 ag(ag_cluster)[909833]: INFO: notify: 2023/07/14 17:21:17 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:21:17 ag(ag_cluster)[909833]: INFO: notify: 2023/07/14 17:21:17 Replica is RESOLVING (0) >Jul 14 17:21:17 ag(ag_cluster)[909833]: INFO: notify: 2023/07/14 17:21:17 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:21:17 ag(ag_cluster)[909833]: INFO: notify: 2023/07/14 17:21:17 Sequence number is 17F:F (1644972474383) >Jul 14 17:21:17 ag(ag_cluster)[909833]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:21:22 ag(ag_cluster)[909833]: INFO: ag_cluster notify : 0 >Jul 14 17:21:22 ag(ag_cluster)[909932]: INFO: mssql_validate >Jul 14 17:21:22 ag(ag_cluster)[909932]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:21:22 ag(ag_cluster)[909932]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:21:22 ag(ag_cluster)[909932]: INFO: Resource agent invoked with: notify >Jul 14 17:21:22 ag(ag_cluster)[909932]: INFO: mssql_notify post-start >Jul 14 17:21:22 ag(ag_cluster)[909932]: INFO: ag_cluster notify : 0 >Jul 14 17:21:23 ag(ag_cluster)[909978]: INFO: mssql_validate >Jul 14 17:21:23 ag(ag_cluster)[909978]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:21:23 ag(ag_cluster)[909978]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:21:23 ag(ag_cluster)[909978]: INFO: Resource agent invoked with: notify >Jul 14 17:21:23 ag(ag_cluster)[909978]: INFO: mssql_notify pre-stop >Jul 14 17:21:23 ag(ag_cluster)[909978]: INFO: ag_cluster notify : 0 >Jul 14 17:21:23 ag(ag_cluster)[910023]: INFO: mssql_validate >Jul 14 17:21:23 ag(ag_cluster)[910023]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:21:23 ag(ag_cluster)[910023]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:21:23 ag(ag_cluster)[910023]: INFO: Resource agent invoked with: notify >Jul 14 17:21:23 ag(ag_cluster)[910023]: INFO: mssql_notify post-stop >Jul 14 17:21:23 ag(ag_cluster)[910023]: INFO: notify: 2023/07/14 17:21:23 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-stop]; connection-timeout [30]; health-threshold [3]; action [post-stop] >Jul 14 17:21:23 ag(ag_cluster)[910023]: INFO: notify: 2023/07/14 17:21:23 ag-helper invoked with required-synchronized-secondaries-to-commit [-1] >Jul 14 17:21:23 ag(ag_cluster)[910023]: INFO: notify: 2023/07/14 17:21:23 Replica is RESOLVING (0) >Jul 14 17:21:23 ag(ag_cluster)[910023]: INFO: ag_cluster notify : 0 >Jul 14 17:21:23 ag(ag_cluster)[910098]: INFO: mssql_validate >Jul 14 17:21:23 ag(ag_cluster)[910098]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:21:23 ag(ag_cluster)[910098]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:21:23 ag(ag_cluster)[910098]: INFO: Resource agent invoked with: notify >Jul 14 17:21:23 ag(ag_cluster)[910098]: INFO: mssql_notify pre-promote >Jul 14 17:21:23 ag(ag_cluster)[910098]: INFO: notify: 2023/07/14 17:21:23 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-promote]; connection-timeout [30]; health-threshold [3]; action [pre-promote] >Jul 14 17:21:23 ag(ag_cluster)[910098]: INFO: notify: 2023/07/14 17:21:23 From RetryExecute - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:21:23 ag(ag_cluster)[910098]: INFO: notify: 2023/07/14 17:21:23 Connected to the instance at localhost:1433 >Jul 14 17:21:28 ag(ag_cluster)[910098]: INFO: notify: 2023/07/14 17:21:28 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:21:28 ag(ag_cluster)[910098]: INFO: notify: 2023/07/14 17:21:28 Sequence number is 17F:F (1644972474383) >Jul 14 17:21:28 ag(ag_cluster)[910098]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:21:33 ag(ag_cluster)[910098]: INFO: ag_cluster notify : 0 >Jul 14 17:21:33 ag(ag_cluster)[910212]: INFO: mssql_validate >Jul 14 17:21:33 ag(ag_cluster)[910212]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:21:33 ag(ag_cluster)[910212]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:21:33 ag(ag_cluster)[910212]: INFO: Resource agent invoked with: notify >Jul 14 17:21:33 ag(ag_cluster)[910212]: INFO: mssql_notify pre-promote >Jul 14 17:21:33 ag(ag_cluster)[910212]: INFO: notify: 2023/07/14 17:21:33 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-pre-promote]; connection-timeout [30]; health-threshold [3]; action [pre-promote] >Jul 14 17:21:33 ag(ag_cluster)[910212]: INFO: notify: 2023/07/14 17:21:33 From RetryExecute - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:21:33 ag(ag_cluster)[910212]: INFO: notify: 2023/07/14 17:21:33 Connected to the instance at localhost:1433 >Jul 14 17:21:38 ag(ag_cluster)[910212]: INFO: notify: 2023/07/14 17:21:38 Replica is SYNCHRONOUS_COMMIT (1). >Jul 14 17:21:38 ag(ag_cluster)[910212]: INFO: notify: 2023/07/14 17:21:38 Sequence number is 17F:F (1644972474383) >Jul 14 17:21:38 ag(ag_cluster)[910212]: INFO: notify: SEQUENCE_NUMBER: 1644972474383 >Jul 14 17:21:43 ag(ag_cluster)[910212]: INFO: ag_cluster notify : 0 >Jul 14 17:21:49 ag(ag_cluster)[910315]: INFO: mssql_validate >Jul 14 17:21:49 ag(ag_cluster)[910315]: INFO: OCF_RESKEY_CRM_meta_interval value: 0 >Jul 14 17:21:49 ag(ag_cluster)[910315]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:21:49 ag(ag_cluster)[910315]: INFO: Resource agent invoked with: notify >Jul 14 17:21:49 ag(ag_cluster)[910315]: INFO: mssql_notify post-promote >Jul 14 17:21:49 ag(ag_cluster)[910315]: INFO: notify: 2023/07/14 17:21:49 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-post-promote]; connection-timeout [30]; health-threshold [3]; action [post-promote] >Jul 14 17:21:49 ag(ag_cluster)[910315]: INFO: notify: 2023/07/14 17:21:49 [DEBUG] AG Helper PostPromote Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:21:49 ag(ag_cluster)[910315]: INFO: notify: 2023/07/14 17:21:49 Replica is RESOLVING (0) >Jul 14 17:21:49 ag(ag_cluster)[910315]: INFO: ag_cluster notify : 0 >Jul 14 17:21:49 ag(ag_cluster)[910390]: INFO: mssql_validate >Jul 14 17:21:49 ag(ag_cluster)[910390]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:21:49 ag(ag_cluster)[910390]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:21:49 ag(ag_cluster)[910390]: INFO: Resource agent invoked with: monitor >Jul 14 17:21:49 ag(ag_cluster)[910390]: INFO: mssql_monitor >Jul 14 17:21:49 ag(ag_cluster)[910390]: INFO: monitor: 2023/07/14 17:21:49 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:21:49 ag(ag_cluster)[910390]: INFO: monitor: 2023/07/14 17:21:49 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:21:49 ag(ag_cluster)[910390]: INFO: monitor: 2023/07/14 17:21:49 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:21:49 ag(ag_cluster)[910390]: INFO: monitor: 2023/07/14 17:21:49 Connected to the instance at localhost:1433 >Jul 14 17:21:54 ag(ag_cluster)[910390]: INFO: monitor: 2023/07/14 17:21:54 Monitor Caller is: monitor. >Jul 14 17:21:54 ag(ag_cluster)[910390]: INFO: monitor: 2023/07/14 17:21:54 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:21:54 ag(ag_cluster)[910390]: INFO: monitor: 2023/07/14 17:21:54 Replica is RESOLVING (0) >Jul 14 17:21:54 ag(ag_cluster)[910390]: INFO: monitor: 2023/07/14 17:21:54 Instance name is server3. >Jul 14 17:21:54 ag(ag_cluster)[910390]: INFO: monitor: 2023/07/14 17:21:54 Current master is server2. >Jul 14 17:21:54 ag(ag_cluster)[910390]: INFO: monitor: 2023/07/14 17:21:54 Setting the role to Secondary. >Jul 14 17:21:54 ag(ag_cluster)[910390]: INFO: monitor: 2023/07/14 17:21:54 Setting replica to SECONDARY role... >Jul 14 17:21:54 ag(ag_cluster)[910390]: INFO: monitor: 2023/07/14 17:21:54 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:21:54 ag(ag_cluster)[910390]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:21:54 ag(ag_cluster)[910390]: INFO: lease_expiry after monitor update: >Jul 14 17:21:54 ag(ag_cluster)[910390]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:21:54 ag(ag_cluster)[910390]: INFO: ag_cluster monitor : 0 >Jul 14 17:21:54 ag(ag_cluster)[910562]: INFO: mssql_validate >Jul 14 17:21:54 ag(ag_cluster)[910562]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:21:54 ag(ag_cluster)[910562]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:21:54 ag(ag_cluster)[910562]: INFO: Resource agent invoked with: monitor >Jul 14 17:21:54 ag(ag_cluster)[910562]: INFO: mssql_monitor >Jul 14 17:21:54 ag(ag_cluster)[910562]: INFO: monitor: 2023/07/14 17:21:54 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:21:54 ag(ag_cluster)[910562]: INFO: monitor: 2023/07/14 17:21:54 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:21:54 ag(ag_cluster)[910562]: INFO: monitor: 2023/07/14 17:21:54 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:21:54 ag(ag_cluster)[910562]: INFO: monitor: 2023/07/14 17:21:54 Connected to the instance at localhost:1433 >Jul 14 17:21:59 ag(ag_cluster)[910562]: INFO: monitor: 2023/07/14 17:21:59 Monitor Caller is: monitor. >Jul 14 17:21:59 ag(ag_cluster)[910562]: INFO: monitor: 2023/07/14 17:21:59 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:21:59 ag(ag_cluster)[910562]: INFO: monitor: 2023/07/14 17:21:59 Replica is RESOLVING (0) >Jul 14 17:21:59 ag(ag_cluster)[910562]: INFO: monitor: 2023/07/14 17:21:59 Instance name is server3. >Jul 14 17:21:59 ag(ag_cluster)[910562]: INFO: monitor: 2023/07/14 17:21:59 Current master is server2. >Jul 14 17:21:59 ag(ag_cluster)[910562]: INFO: monitor: 2023/07/14 17:21:59 Setting the role to Secondary. >Jul 14 17:21:59 ag(ag_cluster)[910562]: INFO: monitor: 2023/07/14 17:21:59 Setting replica to SECONDARY role... >Jul 14 17:21:59 ag(ag_cluster)[910562]: INFO: monitor: 2023/07/14 17:21:59 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:21:59 ag(ag_cluster)[910562]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:21:59 ag(ag_cluster)[910562]: INFO: lease_expiry after monitor update: >Jul 14 17:21:59 ag(ag_cluster)[910562]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:21:59 ag(ag_cluster)[910562]: INFO: ag_cluster monitor : 0 >Jul 14 17:22:06 ag(ag_cluster)[910735]: INFO: mssql_validate >Jul 14 17:22:06 ag(ag_cluster)[910735]: INFO: OCF_RESKEY_CRM_meta_interval value: 12000 >Jul 14 17:22:06 ag(ag_cluster)[910735]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:22:06 ag(ag_cluster)[910735]: INFO: Resource agent invoked with: monitor >Jul 14 17:22:06 ag(ag_cluster)[910735]: INFO: mssql_monitor >Jul 14 17:22:06 ag(ag_cluster)[910735]: INFO: monitor: 2023/07/14 17:22:06 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:22:06 ag(ag_cluster)[910735]: INFO: monitor: 2023/07/14 17:22:06 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [74]; monitor-interval-timeout [60] >Jul 14 17:22:06 ag(ag_cluster)[910735]: INFO: monitor: 2023/07/14 17:22:06 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:22:06 ag(ag_cluster)[910735]: INFO: monitor: 2023/07/14 17:22:06 Connected to the instance at localhost:1433 >Jul 14 17:22:11 ag(ag_cluster)[910735]: INFO: monitor: 2023/07/14 17:22:11 Monitor Caller is: monitor. >Jul 14 17:22:11 ag(ag_cluster)[910735]: INFO: monitor: 2023/07/14 17:22:11 [DEBUG] AG Helper Monitor Role info: AVAILABILITY GROUP ag1 on instance server3 >Jul 14 17:22:11 ag(ag_cluster)[910735]: INFO: monitor: 2023/07/14 17:22:11 Replica is RESOLVING (0) >Jul 14 17:22:11 ag(ag_cluster)[910735]: INFO: monitor: 2023/07/14 17:22:11 Instance name is server3. >Jul 14 17:22:11 ag(ag_cluster)[910735]: INFO: monitor: 2023/07/14 17:22:11 Current master is server2. >Jul 14 17:22:11 ag(ag_cluster)[910735]: INFO: monitor: 2023/07/14 17:22:11 Setting the role to Secondary. >Jul 14 17:22:11 ag(ag_cluster)[910735]: INFO: monitor: 2023/07/14 17:22:11 Setting replica to SECONDARY role... >Jul 14 17:22:11 ag(ag_cluster)[910735]: INFO: monitor: 2023/07/14 17:22:11 Could not set replica to SECONDARY role. Failover Failed. >Jul 14 17:22:11 ag(ag_cluster)[910735]: INFO: monitor: PROMOTION_SCORE: -INFINITY >Jul 14 17:22:11 ag(ag_cluster)[910735]: INFO: lease_expiry after monitor update: >Jul 14 17:22:11 ag(ag_cluster)[910735]: INFO: lease information from all replicas after monitor update: name="ag_cluster-lease-expiry" host="server1" value="" >name="ag_cluster-lease-expiry" host="server3" value="" >name="ag_cluster-lease-expiry" host="server2" value="" >Jul 14 17:22:11 ag(ag_cluster)[910735]: INFO: ag_cluster monitor : 0 >Jul 14 17:22:11 ag(ag_cluster)[910907]: INFO: mssql_validate >Jul 14 17:22:12 ag(ag_cluster)[910907]: INFO: OCF_RESKEY_CRM_meta_interval value: 10000 >Jul 14 17:22:12 ag(ag_cluster)[910907]: INFO: OCF_RESKEY_CRM_meta_timeout value: 60000 >Jul 14 17:22:12 ag(ag_cluster)[910907]: INFO: Resource agent invoked with: monitor >Jul 14 17:22:12 ag(ag_cluster)[910907]: INFO: mssql_monitor >Jul 14 17:22:12 ag(ag_cluster)[910907]: INFO: monitor: 2023/07/14 17:22:12 ag-helper invoked with hostname [localhost]; port [1433]; ag-name [ag1]; credentials-file [/var/opt/mssql/secrets/passwd]; application-name [monitor-ag_cluster-monitor]; connection-timeout [30]; health-threshold [3]; action [monitor] >Jul 14 17:22:12 ag(ag_cluster)[910907]: INFO: monitor: 2023/07/14 17:22:12 ag-helper invoked with required-synchronized-secondaries-to-commit [-1]; current-master [server2]; disable-primary-on-quorum-timeout-after [60]; primary-write-lease-duration [72]; monitor-interval-timeout [60] >Jul 14 17:22:12 ag(ag_cluster)[910907]: INFO: monitor: 2023/07/14 17:22:12 From RetryExecuteWithTimeout - Attempt 1 to connect to the instance at localhost:1433 >Jul 14 17:22:12 ag(ag_cluster)[910907]: INFO: monitor: 2023/07/14 17:22:12 Connected to the instance at localhost:1433
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 2221772
:
1976118
|
1976119
| 1976120 |
1976684
|
1976685
|
1976686