“nsc generate config --nats-resolver” will output extra commas in resolver_preload

[sinbane]

What version were you using?

2.8.6

What environment was the server running in?

k8s

Is this defect reproducible?

yes, just copy direct output of "nsc generate config --nats-resolver"

Given the capability you are leveraging, describe your expectation?

the output of command "nsc generate config --nats-resolver" will be something like below, there's a commas at the end of the account jwt. if someone copies this and use it in config file, will fail to start server. The correct value has no commas.
"

Later changes to the system account take precedence over the system account jwt listed here.

resolver_preload: {
AD46U: eyJ0***JnCQ,
}
"

Given the expectation, what is the defect you are observing?

server fails to start since the incorrect commas

[sinbane]

nsc/cmd/natsresolverconfigbuilder.go

Line 87 in 01db6f2

%s: %s,

[aricart]

If you save it to a file, does it work?

nsc generate config --nats-resolver --config-file /tmp/server.conf

On my config:

# Operator named O
operator: eyJ0eXAiOiJKV1QiLCJhbGciOiJlZDI1NTE5LW5rZXkifQ.eyJqdGkiOiI2SU9aQVVaWkZSS1dHTFk2VVVRTDZZNEFFM1NQU0FWSTZLSFlOTTdIUUZJWUVRTUdGNURRIiwiaWF0IjoxNzE2MzAyNjIzLCJpc3MiOiJPRFJXTFBZTEtIV0VGRVg2TTdTUFRTTE9BS0JJRU1NREZZQ1VWWFNBQlBLQTUyTFhXTDVJNktTSCIsIm5hbWUiOiJPIiwic3ViIjoiT0RSV0xQWUxLSFdFRkVYNk03U1BUU0xPQUtCSUVNTURGWUNVVlhTQUJQS0E1MkxYV0w1STZLU0giLCJuYXRzIjp7InN5c3RlbV9hY2NvdW50IjoiQUQ2M0pFV0E0UTZLVlpZRFBPWFNNS0FKR0tFN1RWN1Q3UFZLT0FNUFlLSUtBRFRRM0VTU1lRT1MiLCJ0eXBlIjoib3BlcmF0b3IiLCJ2ZXJzaW9uIjoyfX0.A8nK5S0anDWB-GbkrN3QMACjoNyt7YtslpwSL4Mj07vfEcWy0s_vxkpr9Q4XvepzNh04R5Em47H48EoynXEJAQ
# System Account named SYS
system_account: AD63JEWA4Q6KVZYDPOXSMKAJGKE7TV7T7PVKOAMPYKIKADTQ3ESSYQOS

# configuration of the nats based resolver
resolver {
    type: full
    # Directory in which the account jwt will be stored
    dir: './jwt'
    # In order to support jwt deletion, set to true
    # If the resolver type is full delete will rename the jwt.
    # This is to allow manual restoration in case of inadvertent deletion.
    # To restore a jwt, remove the added suffix .delete and restart or send a reload signal.
    # To free up storage you must manually delete files with the suffix .delete.
    allow_delete: false
    # Interval at which a nats-server with a nats based account resolver will compare
    # it's state with one random nats based account resolver in the cluster and if needed, 
    # exchange jwt and converge on the same set of jwt.
    interval: "2m"
    # Timeout for lookup requests in case an account does not exist locally.
    timeout: "1.9s"
}


# Preload the nats based resolver with the system account jwt.
# This is not necessary but avoids a bootstrapping system account. 
# This only applies to the system account. Therefore other account jwt are not included here.
# To populate the resolver:
# 1) make sure that your operator has the account server URL pointing at your nats servers.
#    The url must start with: "nats://" 
#    nsc edit operator --account-jwt-server-url nats://localhost:4222
# 2) push your accounts using: nsc push --all
#    The argument to push -u is optional if your account server url is set as described.
# 3) to prune accounts use: nsc push --prune 
#    In order to enable prune you must set above allow_delete to true
# Later changes to the system account take precedence over the system account jwt listed here.
resolver_preload: {
	AD63JEWA4Q6KVZYDPOXSMKAJGKE7TV7T7PVKOAMPYKIKADTQ3ESSYQOS: eyJ0eXAiOiJKV1QiLCJhbGciOiJlZDI1NTE5LW5rZXkifQ.eyJqdGkiOiJDQUFON1BJUEpaQ1dLSDU3Q1dRNEFSMlY0QldLV1VNTEtIRkY1SDNNSU1ZVENLRUhKTVVRIiwiaWF0IjoxNzE2MzAyNjE4LCJpc3MiOiJPRFJXTFBZTEtIV0VGRVg2TTdTUFRTTE9BS0JJRU1NREZZQ1VWWFNBQlBLQTUyTFhXTDVJNktTSCIsIm5hbWUiOiJTWVMiLCJzdWIiOiJBRDYzSkVXQTRRNktWWllEUE9YU01LQUpHS0U3VFY3VDdQVktPQU1QWUtJS0FEVFEzRVNTWVFPUyIsIm5hdHMiOnsibGltaXRzIjp7InN1YnMiOi0xLCJkYXRhIjotMSwicGF5bG9hZCI6LTEsImltcG9ydHMiOi0xLCJleHBvcnRzIjotMSwid2lsZGNhcmRzIjp0cnVlLCJjb25uIjotMSwibGVhZiI6LTF9LCJkZWZhdWx0X3Blcm1pc3Npb25zIjp7InB1YiI6e30sInN1YiI6e319LCJhdXRob3JpemF0aW9uIjp7fSwidHlwZSI6ImFjY291bnQiLCJ2ZXJzaW9uIjoyfX0.oXqaLLf0cIZWKxzLpKSmd48CzaXrakif6g4pA7lRsHg8Bey20zbjRTYZclUi6FzRvuWNYAn4_-TUmYrvk44rBg,
}

aricart@mac-studio /tmp> nats-server -c server.conf
[17360] 2024/05/21 09:46:57.263681 [INF] Starting nats-server
[17360] 2024/05/21 09:46:57.263800 [INF]   Version:  2.10.14
[17360] 2024/05/21 09:46:57.263803 [INF]   Git:      [not set]
[17360] 2024/05/21 09:46:57.263805 [INF]   Name:     NCVXLTOMBHMR464Z7EDYBOMU7DIGU7YM754GZ5X36VGMQKFIBXUGRVYR
[17360] 2024/05/21 09:46:57.263808 [INF]   ID:       NCVXLTOMBHMR464Z7EDYBOMU7DIGU7YM754GZ5X36VGMQKFIBXUGRVYR
[17360] 2024/05/21 09:46:57.263816 [INF] Using configuration file: server.conf
[17360] 2024/05/21 09:46:57.263819 [INF] Trusted Operators
[17360] 2024/05/21 09:46:57.263821 [INF]   System  : ""
[17360] 2024/05/21 09:46:57.263824 [INF]   Operator: "O"
[17360] 2024/05/21 09:46:57.263827 [INF]   Issued  : 2024-05-21 09:43:43 -0500 CDT
[17360] 2024/05/21 09:46:57.263842 [INF]   Expires : Never
[17360] 2024/05/21 09:46:57.264185 [INF] Managing all jwt in exclusive directory /tmp/jwt
[17360] 2024/05/21 09:46:57.264398 [INF] Listening for client connections on 0.0.0.0:4222
[17360] 2024/05/21 09:46:57.264662 [INF] Server is ready

[aricart]

Wondering how you are importing the file

[aricart]

So did some more experiment:

This is a config file that imports auth.conf in the same dir.

cat server.conf
include ./auth.conf

port: 4333

Save the resolver as auth.conf

nsc generate config --nats-resolver --config-file /tmp/auth.conf -F
[ OK ] wrote server configuration to `/tmp/auth.conf`
Success!! - generated `/tmp/auth.conf`

nats-server -c server.conf
[17415] 2024/05/21 09:51:03.628661 [INF] Starting nats-server
[17415] 2024/05/21 09:51:03.628803 [INF]   Version:  2.10.14
[17415] 2024/05/21 09:51:03.628806 [INF]   Git:      [not set]
[17415] 2024/05/21 09:51:03.628808 [INF]   Name:     NBJ5IMX4IA7M6EHIP4E4OKY2HUHCJIZT2YPQ5VGJAQCQSATMLJMNNQ2M
[17415] 2024/05/21 09:51:03.628811 [INF]   ID:       NBJ5IMX4IA7M6EHIP4E4OKY2HUHCJIZT2YPQ5VGJAQCQSATMLJMNNQ2M
[17415] 2024/05/21 09:51:03.628833 [INF] Using configuration file: server.conf
[17415] 2024/05/21 09:51:03.628837 [INF] Trusted Operators
[17415] 2024/05/21 09:51:03.628839 [INF]   System  : ""
[17415] 2024/05/21 09:51:03.628842 [INF]   Operator: "O"
[17415] 2024/05/21 09:51:03.628844 [INF]   Issued  : 2024-05-21 09:43:43 -0500 CDT
[17415] 2024/05/21 09:51:03.628866 [INF]   Expires : Never
[17415] 2024/05/21 09:51:03.629305 [INF] Managing all jwt in exclusive directory /tmp/jwt
[17415] 2024/05/21 09:51:03.629561 [INF] Listening for client connections on 0.0.0.0:4333
[17415] 2024/05/21 09:51:03.629833 [INF] Server is ready

[aricart]

In the above cases the contents for the auth is saved directly to a file, and it is loaded by the server without errors wondering what your setup is doing, as I have the commas in there and it works.

[aricart]

Also what server version are you using?

[sinbane]

Hi aricart, I use helm chart to install on k8s, so I copy the cmd output and use it in value.yaml, then when helm to merge the values, I think it will take the commas as a part of string value to merge. then will meet error when the instance is starting. @aricart

[sinbane]

I use the official helm chart, and install like this "helm upgrade --install nats nats/nats -f nats.yaml".

The nats.yaml is like:

config:
resolver:
enabled: true
merge:
type: full
interval: 2m
timeout: 1.9s
merge:
operator: eyJ0......BDw
system_account: AC5......CK
resolver_preload:
AC5......EICK: eyJ0eX......DA #### if there's comma at the end, will fail to start.

[aricart]

looks like you are assuming that it is yaml, but it is conf, not yaml.

[sinbane]

yes, right, because the doc said nats config is combined benifits of yaml and json, so it will let me think the output config is usable in yaml by mistake. and in the doc of helm chart, there's an example to enable resolver with helm merge, in that example says to copy the output of nsc, so I copied with comma, I thought the comma is some magic char by mistake.

so in my opinion, the comma here is a little useless, but let people confusing.