Im trying to make my vegetable stock into jelly.

Agar agar might be what you're looking for! Often used to make vegetarian soup dumplings (like here: https://theglobalvegetarian.com/recipes/vegan-soup-dumpling).

Your question is right in doubting whether vegetable broth alone (unlike meat-based broth) can make jelly.

So yes, you’ll need to add something like chicken broth or gelatin.

If you want to avoid meat-based gelatin, look among kosher products, where you can find fish and/or plant-based gelatin.