If I need to target specific Ruleset from App Studio when I create a rule, does that mean I should be building a Component?

It may be. It is worth considering, but this is not necessarily the case. 

You may have Rulesets in your Enterprise layer that provide rules that are intended to be Ruleset-specialized by your Application layer, but yet are still part of the reusable Enterprise capability. These may still make sense to live in your Enterprise layer and not be a standalone Component.

Your primary driver for using Components should be whether the feature is "pluggable" into your Application without any Enterprise-layer dependencies.

If you are unsure whether App Studio will pick the correct Ruleset for the rule you are about to create, do it from Dev Studio to get more control over the outcome. Once created, App Studio will typically retain the ruleset from where it found the rule. Using Branch development provides a just-in-time opportunity for the LSA to cross-check that the new/changed rules will be merged into the correct ruleset before the merge happens, enabling revisions to be applied before the merge is complete.

Discussion on this topic was sought from the LSA Data Excellence (Pega 8.4) webinar conducted in July 2020. The webinar and its full set of discussions that arose from it are available at LSA Data Excellence: Webinar, Questions & Answers.