Bug Report: ProposalState creation fails with non-ASCII default locale

Description

When the default locale is set to a non-ASCII language (such as Japanese :ja), the create_default_states! method in decidim-proposals fails to create ProposalState records. This prevents proposal components from having the necessary states (evaluating, accepted, rejected), making proposal answering functionality unavailable in the admin interface.

The issue occurs because the generate_token method in ProposalState cannot properly handle non-ASCII characters in titles when generating tokens, and it incorrectly processes symbol tokens passed from create_default_states!.

Steps to Reproduce

1. Set up a Decidim installation with Japanese as the default locale:

   # config/initializers/decidim.rb
   config.default_locale = :ja
   config.available_locales = [:ja, :en]

2. Create a new proposal component in any participatory space

3. Try to answer a proposal in the admin interface:

   • Go to Admin > [Participatory Space] > Components > Proposals > [Any Proposal]
   • Try to set the proposal state - only "Not answered" (応答がありません) appears

4. Check the database:

   Decidim::Proposals::ProposalState.count # Returns 0

5. Manually try to create default states:

   component = Decidim::Component.where(manifest_name: 'proposals').first
   admin_user = component.organization.admins.first
   Decidim::Proposals.create_default_states!(component, admin_user)

6. Check the created objects - they will have validation errors:

   Errors: ["Token can't be blank"]
   

Expected Behavior

• ProposalState records should be automatically created when a proposals component is created
• The admin interface should show proposal state options (evaluating, accepted, rejected)
• create_default_states! should work regardless of the default locale

Actual Behavior

• No ProposalState records are created
• Admin interface only shows "Not answered" option
• Proposal answering functionality is completely unavailable

Root Cause

File: decidim-proposals/app/models/decidim/proposals/proposal_state.rb

Issue 1: Token generation fails with non-ASCII characters

def generate_token
  self.token = ensure_unique_token(token.presence || translated_attribute(title).parameterize(separator: "_"))
end

When the title is in Japanese (e.g., "承認済み"), parameterize returns an empty string, causing validation failure.

Issue 2: Symbol tokens are not handled properly The create_default_states! method passes tokens as symbols (:accepted), but token.presence doesn't handle symbols correctly, leading to the fallback to the problematic parameterize result.

Environment

• Device: Desktop
• Operating System: macOS 14.5.0
• Browser: Not applicable (server-side issue)
• Decidim Version: 0.29.2
• Decidim Installation: Source code from GitHub

Proposed Fix

Modify the generate_token method to:

1. Properly handle symbol tokens
2. Fall back to English titles or a default when parameterize returns empty

def generate_token
  existing_token = token.is_a?(Symbol) ? token.to_s : token.presence
  fallback_token = translated_attribute(title).parameterize(separator: "_")
  
  # Handle non-ASCII characters by falling back to English or default
  if fallback_token.blank?
    english_title = title.dig("en") || title.dig(:en)
    fallback_token = english_title&.parameterize(separator: "_") || "state"
  end
  
  self.token = ensure_unique_token(existing_token || fallback_token)
end

Impact

This issue affects any Decidim installation using non-ASCII languages as the default locale, making the proposal answering feature completely unusable. This is a critical functionality issue for internationalized Decidim deployments.

Additional Context

This issue was discovered in a Japanese Decidim deployment (decidim-cfj), but likely affects other non-ASCII languages like Chinese, Korean, Arabic, etc.