This procedure may be different from the factory but it is a procedure I have used for many years of installations onto fiberglass. For fasteners I use a good grade. Usually but not always the screws purchased from a marine facility or distribution center are of higher grade than screws purchased from a hardware store or big box. I'm talking good quality stainless steel. A visual look at them tells the story.
The drill size question, there are charts. In most cases you can use a hardwood chart that will work most of the time. Remember all screws are not machined the same. Rule of thumb 1/8" bit for #8 screw and 9/64" for a #10. I have gotten burned a few times in the past with this rule of thumb. stripped or twisted the head of the screw off. The best way to gage the drill bit size is measure the shank of the screw (measure in-between the tip and the head) If the measurement is slightly less than the drill bit size in the index always round up never down. Example: measurement of the shank #8 screw measures .129 the closest drill bit would be 1/8" (.125 diameter) I would not use the 1/8" because it may be to tight and strip the screw head or snap off before it provides proper clamping force. I would use a #29 (.136 diameter) or if fractional bits are indexed a 9/64 (.140) for this #8. Once again the rule of thumb is 1/8" for #8 and 9/64 for #10. I would use a 9/64 for a #8 in this case.
Drilling into fiberglass is not the same as wood. Fiberglass is not as forgiving. Drilling depth must be thru the fiberglass in this application (toe rail mounting). It is always a good idea to make sure there are no wire runs or items that you will drill into. It is not always possible to visually inspect the location. What I do in these cases is mark all the hole locations. I then use a small drill bit to pre-drill each hole. I tape the drill at 3/8" so that I will not drill any deeper then that. Once the hole is drilled though the deck I use the next size smaller drill bit using my fingers to probe the hole to confirm there are no obstructions that I will be screwing into. I then drill proper size holes for mounting. I use a tapered Dremel to cut a taper in the mounting hole to make a relieve so the holes surface doesn't crack ( Gel coat) without cutting this release the screws threads will lift the gelcoat, crack it and cause future cracks or crazing around the hole.
Sealant? I don't use 5200 for something that may need to be replaced in the future. 5200 is a great adhesive sealant but not alway the best for installing components to decks. I prefer to use a good marine silicone sealant (Boatlife Life seal) or 3m fast cure 4200. The application should be a thin layer on the entire toe rail, each hole filled with it and each screw a tab on the threads, for thread lubrication and sealant. After the toe rail is installed and properly fastened there should be evidence of sealant on the outside edges of the toe rail all the way around. I use a wet paper towel to clean any access off so it has a nice finish look.
Good luck with your project.
