Description of problem: In Some IPv6 environment, FRR should not get default router provided by router advertisement. For BGP routing, default route provided by IPv6 RA will prevent the routing provided by BGP. In FRR, local routing is always preferred to received routes. ~~~ Route Selection The route selection process used by FRR’s BGP implementation uses the following decision criterion, starting at the top of the list and going towards the bottom until one of the factors can be used. Weight check Prefer higher local weight routes to lower routes. Local preference check Prefer higher local preference routes to lower. If bgp bestpath aigp is enabled, and both paths that are compared have AIGP attribute, BGP uses AIGP tie-breaking unless both of the paths have the AIGP metric attribute. This means that the AIGP attribute is not evaluated during the best path selection process between two paths when one path does not have the AIGP attribute. Local route check Prefer local routes (statics, aggregates, redistributed) to received routes. ~~~ ref. https://docs.frrouting.org/en/latest/bgp.html To prevent this behavior, we should have a configuration to disable receiving default route by RA in kernel parameter(net.ipv6.conf.all.accept_ra_defrtr = 0 and net.ipv6.conf.default.accept_ra_defrtr = 0) and remove default route once FRR receives BGP routing.
Hello Keigo, If you want to set those accept_ra sysctls to 0 you can simply do so via https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/17.0/html-single/director_installation_and_usage/index#proc_increase-conntrack-entries_additional-network-configuration either globally: parameter_defaults: ExtraSysctlSettings: net.ipv6.conf.all.accept_ra_defrtr: value: 0 net.ipv6.conf.default.accept_ra_defrtr: value: 0 or per role, using: parameter_defaults: <role>Parameters: ExtraSysctlSettings: ... would that help? We can't set those sysctls to zero by default (in tht or similar) without risking to break existing installations. Regards, Luca
Hi Luca, Thank you for your comment. Yes. Applying the kernel parameter require the additional caution. 1. During the deployment the node should use default route until BGP routing is received. 2. Once BGP routing is received and the BGP routing should be preferred to default routing, we should apply the kernel parameter and remove the default routing. Is it possible to provide a new option to enable this behavior? I'd like to hear your thoughts regarding this. Best regards, Keigo Noha
Doing what you propose would require orchestration between initial network setup, frr/bgp configuration and subsequent additional network tweaks to achieve the desired state. To my knowledge we don't have such orchestrations capabilities in tripleo and it is unlikely we will be able to implement something like this, so I think we could to do something like: 1. set up a temporary default route using FirstBoot or ExtraConfigPre (https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/extra_config.html) 2. use ExtraConfigPost to remove that route and add the necessary sysctls (if you don't want to add them via ExtraSysctlSettings as suggested previously) example taken from our lab where we use the same trick. We used ipv4 but it would work with ipv6 as well: ~~~ $ cat extraconfigpre_template.yaml heat_template_version: newton description: > Inject stuff before puppet kicks in parameters: server: type: string resources: CustomExtraConfigPre: type: OS::Heat::SoftwareConfig properties: group: script config: | #!/bin/sh ip route add 10.0.0.0/8 via 192.168.$(hostname |cut -d. -f1 |cut -d- -f2).1 CustomExtraDeploymentPre: type: OS::Heat::SoftwareDeployment properties: server: {get_param: server} config: {get_resource: CustomExtraConfigPre} actions: ['CREATE','UPDATE'] outputs: deploy_stdout: description: Deployment reference, used to trigger pre-deploy on changes value: {get_attr: [CustomExtraDeploymentPre, deploy_stdout]} ~~~ ~~~ $ cat extraconfigpost_template.yaml heat_template_version: newton description: > Inject stuff at the end of the deployment parameters: servers: type: json DeployIdentifier: type: string EndpointMap: default: {} description: Mapping of service endpoint -> protocol. Typically set via parameter_defaults in the resource registry. type: json resources: CustomExtraConfig: type: OS::Heat::SoftwareConfig properties: group: script config: | #!/bin/sh # remove route sysctl net.ipv6.conf.all.accept_ra_defrtr=0 sysctl net.ipv6.conf.default.accept_ra_defrtr=0 ip r del 10.0.0.0/8 CustomExtraDeployments: type: OS::Heat::SoftwareDeploymentGroup properties: config: {get_resource: CustomExtraConfig} servers: {get_param: servers} actions: ['CREATE','UPDATE'] input_values: deploy_identifier: {get_param: DeployIdentifier} ~~~ here we used actions: ['CREATE','UPDATE'] but I think it would be more appropriate to use actions: ['CREATE'] so it doesn't run on stack updates, scale ups etc. Alternatively they could try using setting a route with a weight value such that it would be de-prioritized as soon as the new routes injected via bgp are present. I haven't tested this last option though. HTH Luca