Skip to content

Instantly share code, notes, and snippets.

@MohamedBrary

MohamedBrary/faker.rake

Last active May 1, 2017 15:07
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save MohamedBrary/40f09d48a8e5ee6bda5f639402271c95 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save MohamedBrary/40f09d48a8e5ee6bda5f639402271c95 to your computer and use it in GitHub Desktop.
Download ZIP
This is a rake file having multiple tasks used to generate relational data for a very complicated system, it uses "Faker" and "Populator" to generate database records using real data efficiently, so different benchmark tests and analysis can be done. It is project specific in term of data relations, but the general idea of generating huge amount…
Raw
faker.rake
# ---------- Fakers
# rake fake:cluster - generates the whole cluster data
# rake fake:cluster_without_evaluations - generates fake cluster data but without evaluations (no templates or template_answers)
# rake fake:evaluations - generates evaluations against the existed templates
# ---------- Documentation
# Before you try this faker, it is better to create another db and use it only when u need to benchmark something or check the performance.
# The main rake creates a proxy, and couple of servers, couple of roles and groups, create supervisors and dozens of agents, assign access for each object, and creates templates of questions.
# for each agent it generates a handful of calls, randomly evaluate these calls with double templates, calibration or single evaluation. also it generates random call notes on some random calls.
# all the numbers are at the beginning of the rake, u can edit them as u like, two sets of numbers are in the code, one generates 225,000 calls and the other generates 22,500.
# it takes time so don't go crazy with numbers.
if defined?(Faker)
require 'ffaker'
require 'populator'
# gem "colorize" is required for coloring console output
desc 'Load fake data for development/testing.'
task :clean_db => ['db:drop', 'db:create', 'db:migrate', 'db:seed']
task :faker => ['fake:cluster_without_evaluations', 'fake:templates', 'fake:full_evaluations']
task :large_faker => ['fake:cluster_without_evaluations', 'fake:templates', 'fake:full_evaluations_optimized']
task :mediaum_faker => ['fake:cluster_without_evaluations', 'fake:templates', 'fake:evaluations_optimized']
task :clean_data => ['clean_db', 'faker']
task :clean_large_data => ['clean_db', 'large_faker']
debug = true
namespace :fake do
desc 'Create full cluster data'
task :cluster => :environment do
user_password = '123123123'
departments = ['Technical', 'Relations', 'Marketing', 'Manners', 'Duration']
num_general_templates = departments.length
extension_start_value = 1000
agent_id_start_value = 3000
percentage_calibration = 1
percentage_double_evaluation = 1
percentage_evaluation = 1
date_span = 1 # in years
# 150,000 calls
# 800 users
num_max_questions = 1
num_servers = 5
num_supervisors_per_group = 10
num_agents_per_supervisor = 15
num_calls_per_agent = 200
# 225,000 calls
# 800 users
# num_max_questions = 1
# num_servers = 5
# num_supervisors_per_group = 10
# num_agents_per_supervisor = 15
# num_calls_per_agent = 300
# 22,500 calls
# 800 users
# num_max_questions = 1
# num_servers = 5
# num_supervisors_per_group = 10
# num_agents_per_supervisor = 15
# num_calls_per_agent = 30
# 3,000,000 calls
# 2100 users
# num_max_questions = 1
# num_servers = 5
# num_supervisors_per_group = 20
# num_agents_per_supervisor = 20
# num_calls_per_agent = 1500
# refernece call to copy its media file data
refernece_call = {mp4_file: 'DEMO/DEMO.MP4', audio_file: 'DEMO/DEMO.MP3', media_type: 3}
# retrieving super user data
super_user = User.super_user
super_role = Role.super_role
# creating the cluster
proxy_prefix, proxy = create_cluster(debug)
# creating general templates
puts "creating general templates".colorize(:green)
general_templates = []
(1..num_general_templates).each do |index|
# creating the template
general_template = Template.create!( {user_id: super_user.id, title: "#{proxy_prefix} #{departments[index-1]}"} )
# creating the questions section
questions_section = general_template.questions_sections.create!( {title: 'Section 1', rank: 100} )
# creating random number of yes/no questions
(1..rand(num_max_questions)+2).each do |question_index|
QuestionYesNo.create!( {template_id: general_template.id, questions_section_id: questions_section.id, text: "Question #{question_index}: #{Populator.words(3..5)} ?", coach_hint: Populator.sentences(0..1), default_response: QuestionYesNo.list_possible_answers.sample, weight: rand(5)*10, passing: rand(5)*10, rank: question_index} )
end
# assign super user access to it
RoleAccess.create!( {role_id: super_role.id, accessable_type: 'Template', accessable_id: general_template.id} )
general_templates << general_template
end # of creating generate template
agent_id_counter = agent_id_start_value
# creating servers and their dependant data
(1..num_servers).each do |servers_index|
puts "creating server #{servers_index}".colorize(:blue)
# creating the server
server_prefix, server = create_server(debug, proxy)
extension_counter = extension_start_value
# creating management role and group
puts "creating management role and group".colorize(:green)
management_role = Role.create!({name: "#{server_prefix} Management", supervisable: true})
management_group = Group.create!({name: "#{server_prefix} Management", role_ids: management_role.id})
# creating agents role and group
puts "creating agents role and group".colorize(:green)
agent_role = Role.create!({name: "#{server_prefix} Agent", access_own_data: true})
agent_group = Group.create!({name: "#{server_prefix} Agents", role_ids: agent_role.id})
RoleAccess.create!( {role_id: management_role.id, accessable_type: 'Group', accessable_id: agent_group.id} )
# creating server template
puts "creating server template".colorize(:green)
server_template = Template.create!( {user_id: super_user.id, title: "#{server_prefix} Template"} )
questions_section = server_template.questions_sections.create!( {title: 'Section 1', rank: 100} )
(1..rand(num_max_questions)+2).each do |question_index|
QuestionYesNo.create!( {template_id: server_template.id, questions_section_id: questions_section.id, text: "Question #{question_index}: #{Populator.words(3..5)} ?", coach_hint: Populator.sentences(0..1), default_response: QuestionYesNo.list_possible_answers.sample, weight: rand(5)*10, passing: rand(5)*10, rank: question_index} )
end
RoleAccess.create!( {role_id: management_role.id, accessable_type: 'Template', accessable_id: server_template.id} )
# creating supervisors
puts "creating #{num_supervisors_per_group} supervisors".colorize(:green)
supervisors = []
(1..num_supervisors_per_group).each do |supervisor_index|
puts "creating supervisor #{supervisor_index} of server #{servers_index}".colorize(:cyan) if debug
# incrementing the agent_id counter
agent_id_counter += 1
# department data
department_index = supervisor_index % num_general_templates
department = departments[department_index]
department_template = general_templates[department_index]
# create supervisor
full_name = Faker::Name.name
supervisor = User.create!( {
full_name: full_name,
first_name: full_name.split.first,
last_name: full_name.split.last,
group_id: management_group.id,
password: user_password,
password_confirmation: user_password,
email: Faker::Internet.email,
department: department,
inherit_role: true,
inherit_group_access: true,
inherit_role_templates: true,
phone: Faker::PhoneNumber.phone_number,
agent_id: agent_id_counter.to_s,
created_at: date_span.year.ago - rand(100).day
} )
RoleAccess.create!( {role_id: management_role.id, accessable_type: 'User', accessable_id: supervisor.id} )
supervisors << supervisor
# create subordinates for the supervisor
(1..num_agents_per_supervisor).each do |agent_index|
puts "creating agent(#{agent_index}) for supervisor(#{supervisor_index}), server (#{servers_index})".colorize(:red) if debug
# incrementing the agent_id counter
agent_id_counter += 1
# create agent's extension
extension = server.extensions.create!( {extension_id: extension_counter} )
extension_counter += 1
# create agent
full_name = Faker::Name.name
agent = User.create!( {
full_name: full_name,
first_name: full_name.split.first,
last_name: full_name.split.last,
group_id: agent_group.id,
supervisor_id: supervisor.id,
extension_id: extension.id,
password: user_password,
password_confirmation: user_password,
email: Faker::Internet.email,
department: department,
inherit_role: true,
inherit_group_access: true,
inherit_role_templates: true,
phone: Faker::PhoneNumber.phone_number,
agent_id: agent_id_counter.to_s,
created_at: date_span.year.ago - rand(50).day
} )
RoleAccess.create!( {role_id: management_role.id, accessable_type: 'User', accessable_id: agent.id} )
# populate call data
call_index = -1
Call.populate num_calls_per_agent do |call|
call_index += 1
# puts "creating call (#{call_index}) for agent(#{agent_index})" if debug
call.user_id = agent.id
call.agent_id = agent.agent_id
call.server_id = server.id
call.extension_id = extension.id
call.call_guid = "#{server_prefix}:#{agent.id}:#{call_index}"
call.caller_id = '3216973800'
call.call_id = 11111111..99999999
call.call_direction = ['Inbound', 'Outbound']
call.call_duration = 30..700
call.media_file_size = 500..10000
call.mp4_file = refernece_call[:mp4_file]
call.audio_file = refernece_call[:audio_file]
call.media_type = refernece_call[:media_type]
call.caller_id = Faker::PhoneNumber.phone_number
call.state = 'completed'
call.audio_recording_time = date_span.year.ago + rand(date_span*360).day
call.created_at = call.audio_recording_time + rand(360).minute
# create max two call notes per call
call.call_notes_count = 0
CallNote.populate rand(2) do |call_note|
call_note.user_id = [supervisor.id, agent.id]
call_note.call_id = call.id
call_note.note = Populator.words(4..10)
call_note.created_at = call.created_at..Time.now
call.call_notes_count += 1
end
# flip coins to create calibration
if supervisor_index > 1 && rand(100 / percentage_calibration) == 1
puts "creating calibration".colorize(:green) if debug
template = [department_template, server_template].sample
other_supervisor = supervisors[rand(supervisor_index-1)]
template_index = -1
TemplateAnswer.populate 2 do |template_answer|
template_index += 1
template_answer.user_id = [other_supervisor.id, supervisor.id][template_index]
template_answer.call_id = call.id
template_answer.template_id = template.id
template_answer.for_calibration = true
template_answer.created_at = call.created_at..Time.now
# answer one random question only
question = template.questions.sample
QuestionAnswer.populate 1 do |question_answer|
question_answer.user_id = [other_supervisor.id, supervisor.id][template_index]
question_answer.question_id = question.id
question_answer.template_answer_id = template_answer.id
question_answer.answer = question.list_possible_answers.sample
question_answer.weight = question.weigh_answer(question_answer)
question_answer.created_at = template_answer.created_at
template_answer.total_weight = question.weight
template_answer.answer_weight = question_answer.weight
template_answer.score_percentage = template_answer.total_weight == 0 ? 0 : (template_answer.answer_weight.to_f / template_answer.total_weight.to_f * 100)
end # of question answer
end # of template answer
# flip coins to create evaluation
elsif rand(100 / percentage_evaluation) == 1
puts "creating evaluation".colorize(:green) if debug
template = [department_template, server_template].sample
TemplateAnswer.populate 1 do |template_answer|
template_answer.user_id = supervisor.id
template_answer.call_id = call.id
template_answer.template_id = template.id
template_answer.for_calibration = false
template_answer.created_at = call.created_at..Time.now
question = template.questions.sample
QuestionAnswer.populate 1 do |question_answer|
question_answer.user_id = supervisor.id
question_answer.question_id = question.id
question_answer.template_answer_id = template_answer.id
question_answer.answer = question.list_possible_answers.sample
question_answer.weight = question.weigh_answer(question_answer)
question_answer.created_at = template_answer.created_at
template_answer.total_weight = question.weight
template_answer.answer_weight = question_answer.weight
template_answer.score_percentage = template_answer.total_weight == 0 ? 0 : (template_answer.answer_weight.to_f / template_answer.total_weight.to_f * 100)
end # of question answer
end # of template answer
# flip coins to create double evaluations
elsif rand(100 / percentage_double_evaluation) == 1
puts "creating double evaluations".colorize(:green) if debug
template = [department_template, server_template].sample
other_supervisor = supervisors[rand(supervisor_index)]
template_index = -1
TemplateAnswer.populate 2 do |template_answer|
template_index += 1
template_answer.user_id = [other_supervisor.id, supervisor.id][template_index]
template_answer.template_id = [department_template, server_template][template_index]
template_answer.call_id = call.id
template_answer.for_calibration = false
template_answer.created_at = call.created_at..Time.now
# answer one random question only
question = template.questions.sample
QuestionAnswer.populate 1 do |question_answer|
question_answer.user_id = [other_supervisor.id, supervisor.id][template_index]
question_answer.question_id = question.id
question_answer.template_answer_id = template_answer.id
question_answer.answer = question.list_possible_answers.sample
question_answer.weight = question.weigh_answer(question_answer)
question_answer.created_at = template_answer.created_at
template_answer.total_weight = question.weight
template_answer.answer_weight = question_answer.weight
template_answer.score_percentage = template_answer.total_weight == 0 ? 0 : (template_answer.answer_weight.to_f / template_answer.total_weight.to_f * 100)
end # of question answer
end # of template answer
end # of calibration and evaluation
end # of calls
end # of agent
end # of supervisor
# calculate server evaluations score
# puts "calculate #{server_template.template_answers.count} server evaluations score".colorize(:green)
# server_template.template_answers.each do |template_answer|
# template_answer.calculate_score
# template_answer.save
# end # of calculating this server evaluations score
end # of server
# calculate general templates evaluations score
# puts "calculate general templates evaluations score".colorize(:green) if debug
# general_templates.each do |general_template|
# puts "calculate #{general_template.template_answers.count} general evaluations score".colorize(:green)
# general_template.template_answers.each do |template_answer|
# template_answer.calculate_score
# template_answer.save
# end
# end # of calculating general templates evaluations score
# reset caches
# Call.reset_column_information
end # of faking cluster
desc 'Create cluster data without evaluations'
task :cluster_without_evaluations => :environment do
user_password = '123123123'
departments = ['Technical', 'Relations', 'Marketing', 'Manners', 'Duration']
num_general_templates = departments.length
extension_start_value = 1000
agent_id_start_value = 3000
percentage_calibration = 1
percentage_double_evaluation = 1
percentage_evaluation = 1
date_span = 2 # in years
# 450,000 calls
# 1200 users
# num_max_questions = 1
# num_servers = 1
# num_supervisors_per_group = 1
# num_agents_per_supervisor = 10
# num_calls_per_agent = 20
# 5,000,000 calls
# 200 users
# num_max_questions = 1
# num_servers = 5
# num_supervisors_per_group = 5
# num_agents_per_supervisor = 8
# num_calls_per_agent = 25000
# 150,000 calls
# 800 users
# num_max_questions = 1
# num_servers = 5
# num_supervisors_per_group = 10
# num_agents_per_supervisor = 15
# num_calls_per_agent = 200
# 225,000 calls
# num_max_questions = 1
# num_servers = 5
# num_supervisors_per_group = 10
# num_agents_per_supervisor = 15
# num_calls_per_agent = 300
# 22,500 calls
# 800 users
num_max_questions = 1
num_servers = 3
num_supervisors_per_group = 10
num_agents_per_supervisor = 15
num_calls_per_agent = 30
# 3,000,000 calls
# 2100 users
# num_max_questions = 1
# num_servers = 2
# num_supervisors_per_group = 20
# num_agents_per_supervisor = 20
# num_calls_per_agent = 1500
# refernece call to copy its media file data
refernece_call = {mp4_file: 'DEMO/DEMO.MP4', audio_file: 'DEMO/DEMO.MP3', media_type: 3}
# retrieving super user data
super_user = User.super_user
super_role = Role.super_role
# creating the cluster
puts "creating the cluster".colorize(:green)
proxy_name = Faker::Company.name.split(/[,;-]|\s/).reject{|s| s == '' or s == 'and'}[0..1].join(' ')
proxy_prefix = proxy_name.split(/[,;]|\s/).first
proxy = Proxy.create!( {name: proxy_name, qc_primary_ip: '172.16.20.110', qc_primary_port: '3010', proxy_primary_ip: '172.16.20.108', proxy_primary_port: '8000'} )
agent_id_counter = agent_id_start_value
# creating servers and their dependant data
(1..num_servers).each do |servers_index|
puts "creating server #{servers_index}".colorize(:blue)
# creating the server
server_name = Faker::Company.name.split(/[,;-]|\s/).reject{|s| s == '' or s == 'and'}[0..1].join(' ')
server_prefix = server_name.split(/[,;]|\s/).first
server = proxy.servers.create!( {name: server_name} )
extension_counter = extension_start_value
# creating management role and group
puts "creating management role and group".colorize(:green)
management_role = Role.create!({name: "#{server_prefix} Management", supervisable: true})
management_group = Group.create!({name: "#{server_prefix} Management", role_ids: management_role.id})
# creating agents role and group
puts "creating agents role and group".colorize(:green)
agent_role = Role.create!({name: "#{server_prefix} Agent", access_own_data: true})
agent_group = Group.create!({name: "#{server_prefix} Agents", role_ids: agent_role.id})
RoleAccess.create!( {role_id: management_role.id, accessable_type: 'Group', accessable_id: agent_group.id} )
# creating supervisors
puts "creating #{num_supervisors_per_group} supervisors".colorize(:green)
supervisors = []
(1..num_supervisors_per_group).each do |supervisor_index|
puts "creating supervisor #{supervisor_index}".colorize(:cyan) if debug
# incrementing the agent_id counter
agent_id_counter += 1
# department data
department_index = supervisor_index % num_general_templates
department = departments[department_index]
# create supervisor
full_name = Faker::Name.name
supervisor = User.create!( {
full_name: full_name,
first_name: full_name.split.first,
last_name: full_name.split.last,
group_id: management_group.id,
password: user_password,
password_confirmation: user_password,
email: Faker::Internet.email,
department: department,
inherit_role: true,
inherit_group_access: true,
inherit_role_templates: true,
phone: Faker::PhoneNumber.phone_number,
agent_id: agent_id_counter.to_s,
created_at: date_span.year.ago - rand(100).day
} )
RoleAccess.create!( {role_id: management_role.id, accessable_type: 'User', accessable_id: supervisor.id} )
supervisors << supervisor
# create subordinates for the supervisor
(1..num_agents_per_supervisor).each do |agent_index|
puts "creating agent(#{agent_index}) for supervisor(#{supervisor_index}), server (#{servers_index})".colorize(:red) if debug
# incrementing the agent_id counter
agent_id_counter += 1
# create agent's extension
extension = server.extensions.create!( {extension_id: extension_counter} )
extension_counter += 1
# create agent
full_name = Faker::Name.name
agent = User.create!( {
full_name: full_name,
first_name: full_name.split.first,
last_name: full_name.split.last,
group_id: agent_group.id,
supervisor_id: supervisor.id,
extension_id: extension.id,
password: user_password,
password_confirmation: user_password,
email: Faker::Internet.email,
department: department,
inherit_role: true,
inherit_group_access: true,
inherit_role_templates: true,
phone: Faker::PhoneNumber.phone_number,
agent_id: agent_id_counter.to_s,
created_at: date_span.year.ago - rand(50).day
} )
RoleAccess.create!( {role_id: management_role.id, accessable_type: 'User', accessable_id: agent.id} )
# populate call data
call_index = -1
Call.populate num_calls_per_agent do |call|
call_index += 1
# puts "creating call (#{call_index}) for agent(#{agent_index})" if debug
call.user_id = agent.id
call.agent_id = agent.agent_id
call.server_id = server.id
call.extension_id = extension.id
call.call_guid = "#{server_prefix}:#{agent.id}:#{call_index}"
call.caller_id = '3216973800'
call.call_id = 11111111..99999999
call.call_direction = ['Inbound', 'Outbound']
call.call_duration = 30..700
call.media_file_size = 500..10000
call.mp4_file = refernece_call[:mp4_file]
call.audio_file = refernece_call[:audio_file]
call.media_type = refernece_call[:media_type]
call.caller_id = Faker::PhoneNumber.phone_number
call.state = 'completed'
call.audio_recording_time = date_span.year.ago + rand(date_span*360).day
call.created_at = call.audio_recording_time + rand(360).minute
# create max two call notes per call
call.call_notes_count = 0
CallNote.populate rand(2) do |call_note|
call_note.user_id = [supervisor.id, agent.id]
call_note.call_id = call.id
call_note.note = Populator.words(4..10)
call_note.created_at = call.created_at..Time.now
call.call_notes_count += 1
end
end # of calls
end # of agent
end # of supervisor
end # of server
# reset caches
# Call.reset_column_information
end # of faking cluster without evaluations
desc 'Fake evaluations'
task :evaluations => :environment do
# As max. expected numbers are 200k evaluations for 5M calls ~4%
# Double this number as lots of evaluation would be cut out to mimic the average score percentage to around 85-90%
num_evaluations = (Call.count * 0.04 * 1.5).to_i
min_call_id = Call.minimum('id')
max_call_id = Call.maximum('id')
templates = Template.all
TemplateAnswer.populate num_evaluations do |template_answer|
# pick a random call
begin
random_call_id = min_call_id + rand(max_call_id)
call = Call.find_by_id(random_call_id)
end while call.blank? || call.user.supervisor.blank?
# generate a type of evaluation: calibration, double evaluations, or single evaluation
template_answer = create_type_of_evaluation(debug, template_answer, call, templates)
end # end of populating evaluations
# adjusting the average percentage
adjust_average_score(debug)
end # of faking evaluations
desc 'Fake evaluation for each call'
task :full_evaluations => :environment do
num_evaluations_per_call = 1
templates = Template.all
# looping on all calls and evaluating them
Call.find_each do |call|
TemplateAnswer.populate num_evaluations_per_call do |template_answer|
# generate a type of evaluation: calibration, double evaluations, or single evaluation
template_answer = create_type_of_evaluation(debug, template_answer, call, templates)
end # of populating evaluations
end # of looping on calls
# adjusting the average percentage
adjust_average_score(debug)
end # of faking full_evaluations
desc 'Fake evaluation for percentage of calls'
task :evaluations_optimized => :environment do
evaluations_percentage = 20
num_evaluations = Call.count * evaluations_percentage / 100
call_step = Call.count / num_evaluations
templates = Template.all
percentage_na_questions = 10
percentage_random_answer = 25
# looping on all calls and evaluating them
call_index = 1
evals_count = 0
TemplateAnswer.populate num_evaluations do |template_answer|
evals_count += 1
call = Call.find(call_index)
call_index += call_step
# choose the call's user supervisor or pick random supervisors
supervisor = call.user.supervisor
# pick a random template, their number is limited
template = templates.sample
# print status each 5000 call
puts "creating evaluation #{evals_count}".colorize(:green) if debug && (evals_count % 5000 == 0)
# create first evaluation
template_answer.user_id = supervisor.id
template_answer.call_id = call.id
template_answer.template_id = template.id
template_answer.for_calibration = true
template_answer.created_at = call.created_at + rand(3).day # after creating the call by max. 3 days
template_answer.total_weight = 0
template_answer.answer_weight = 0
# pick the questions to be answered
questions = template.parent_questions.select{ |q| q unless (rand(100 / percentage_na_questions) == 1)}
questions_index = -1
QuestionAnswer.populate questions.count do |question_answer|
questions_index += 1
question = questions[questions_index]
question_answer.user_id = supervisor.id
question_answer.question_id = question.id
question_answer.template_answer_id = template_answer.id
question_answer.created_at = template_answer.created_at
# giving weight to highest weight answers
if (rand(100 / percentage_random_answer) != 1)
h = question.list_possible_answers_with_weights
answer = Hash[h.sort_by{ |_, v| -v }].to_a[rand(1)].first
else
answer = question.list_possible_answers.sample
end
question_answer.answer = answer
# calculating answer weight
question_answer.weight = question.weigh_answer(question_answer)
template_answer.total_weight += question.weight
template_answer.answer_weight += question_answer.weight
end # of question answer
template_answer.score_percentage = template_answer.total_weight == 0 ? 0 : (template_answer.answer_weight.to_f / template_answer.total_weight.to_f * 100)
end # of populating evaluations
# adjusting the average percentage
adjust_average_score(debug)
puts "adding initial values of the template answers, this would take a while for large database"
Template.reset_column_information
Template.pluck(:id).each do |template_id|
# template_answers_count is in readonly list, thus needs the special method reset_counters
Template.reset_counters(template_id, :template_answers)
end
puts "adding initial values of the question answers, this would take a while for large database"
Question.reset_column_information
total_questions = Question.count
Question.pluck(:id).each do |question_id|
# question_answers_count is in readonly list, thus needs the special method reset_counters
puts "Question #{question_id}/#{total_questions}"
Question.reset_counters(question_id, :question_answers)
end
end # of faking evaluations_optimized
desc 'Fake evaluation for each call'
task :full_evaluations_optimized => :environment do
num_evaluations_per_call = 1
templates = Template.all
percentage_na_questions = 10
percentage_random_answer = 25
# looping on all calls and evaluating them
call_index = 0
TemplateAnswer.populate Call.count do |template_answer|
call_index += 1
call = Call.find(call_index)
# choose the call's user supervisor or pick random supervisors
supervisor = call.user.supervisor
# pick a random template, their number is limited
template = templates.sample
# print status each 1000 call
puts "creating evaluation #{call_index}".colorize(:green) if debug && (call_index % 5000 == 0)
# create first evaluation
template_answer.user_id = supervisor.id
template_answer.call_id = call.id
template_answer.template_id = template.id
template_answer.for_calibration = true
template_answer.created_at = call.created_at + rand(3).day # after creating the call by max. 3 days
template_answer.total_weight = 0
template_answer.answer_weight = 0
# pick the questions to be answered
questions = template.parent_questions.select{ |q| q unless (rand(100 / percentage_na_questions) == 1)}
questions_index = -1
QuestionAnswer.populate questions.count do |question_answer|
questions_index += 1
question = questions[questions_index]
question_answer.user_id = supervisor.id
question_answer.question_id = question.id
question_answer.template_answer_id = template_answer.id
question_answer.created_at = template_answer.created_at
# giving weight to highest weight answers
if (rand(100 / percentage_random_answer) != 1)
h = question.list_possible_answers_with_weights
answer = Hash[h.sort_by{ |_, v| -v }].to_a[rand(1)].first
else
answer = question.list_possible_answers.sample
end
question_answer.answer = answer
# calculating answer weight
question_answer.weight = question.weigh_answer(question_answer)
template_answer.total_weight += question.weight
template_answer.answer_weight += question_answer.weight
end # of question answer
template_answer.score_percentage = template_answer.total_weight == 0 ? 0 : (template_answer.answer_weight.to_f / template_answer.total_weight.to_f * 100)
end # of populating evaluations
# adjusting the average percentage
adjust_average_score(debug)
puts "adding initial values of the template answers, this would take a while for large database"
Template.reset_column_information
Template.pluck(:id).each do |template_id|
# template_answers_count is in readonly list, thus needs the special method reset_counters
Template.reset_counters(template_id, :template_answers)
end
puts "adding initial values of the question answers, this would take a while for large database"
Question.reset_column_information
total_questions = Question.count
Question.pluck(:id).each do |question_id|
# question_answers_count is in readonly list, thus needs the special method reset_counters
puts "Question #{question_id}/#{total_questions}"
Question.reset_counters(question_id, :question_answers)
end
end # of faking full_evaluations_optimized
desc 'Fake templates'
task :templates => :environment do
# import the templates saved in 'lib/tasks/faker_templates'
files = Dir.glob("lib/tasks/faker_templates/*.xls")
files.each do |template_file|
Template.import!(File.new(template_file), User.super_user)
end
puts "Imported #{files.count} templates successfully!".colorize(:cyan)
end # of faking templates
#-------------- Utility Functions
def create_cluster(debug)
puts "creating the cluster".colorize(:green)
proxy_name = Faker::Company.name.split(/[,;-]|\s/).reject{|s| s == '' or s == 'and'}[0..1].join(' ')
proxy_prefix = proxy_name.split(/[,;]|\s/).first
proxy = Proxy.create!( {name: proxy_name, qc_primary_ip: '172.16.20.110', qc_primary_port: '3010', proxy_primary_ip: '172.16.20.108', proxy_primary_port: '8000'} )
[proxy_prefix, proxy]
end
def create_server(debug, proxy)
server_name = Faker::Company.name.split(/[,;-]|\s/).reject{|s| s == '' or s == 'and'}[0..1].join(' ')
server_prefix = server_name.split(/[,;]|\s/).first
server = proxy.servers.create!( {name: server_name} )
[server_prefix, server]
end
def create_evaluation(debug, template_answer, call, template, audtior)
percentage_na_questions = 10
percentage_random_answer = 25
# create first evaluation
template_answer.user_id = audtior.id
template_answer.call_id = call.id
template_answer.template_id = template.id
template_answer.for_calibration = true
template_answer.created_at = call.created_at + rand(3).day # after creating the call by max. 3 days
template_answer.total_weight = 0
template_answer.answer_weight = 0
# pick the questions to be answered
questions = template.parent_questions.select{ |q| q unless (rand(100 / percentage_na_questions) == 1)}
questions_index = -1
QuestionAnswer.populate questions.count do |question_answer|
questions_index += 1
question = questions[questions_index]
question_answer.user_id = audtior.id
question_answer.question_id = question.id
question_answer.template_answer_id = template_answer.id
question_answer.created_at = template_answer.created_at
# giving weight to highest weight answers
if (rand(100 / percentage_random_answer) != 1)
h = question.list_possible_answers_with_weights
answer = Hash[h.sort_by{ |_, v| -v }].to_a[rand(1)].first
else
answer = question.list_possible_answers.sample
end
question_answer.answer = answer
# calculating answer weight
question_answer.weight = question.weigh_answer(question_answer)
template_answer.total_weight += question.weight
template_answer.answer_weight += question_answer.weight
end # of question answer
template_answer.score_percentage = template_answer.total_weight == 0 ? 0 : (template_answer.answer_weight.to_f / template_answer.total_weight.to_f * 100)
end
def create_calibration(debug, template_answer, call, template, audtior, second_auditor)
# create first evaluation
template_answer = create_evaluation(debug, template_answer, call, template, audtior)
# create second evaluation
TemplateAnswer.populate 1 do |other_template_answer|
other_template_answer = create_evaluation(debug, other_template_answer, call, template, second_auditor)
end
return template_answer
end
def create_double_evaluations(debug, template_answer, call, template, audtior, second_template, second_auditor)
# create first evaluation
template_answer = create_evaluation(debug, template_answer, call, template, audtior)
# create second evaluation
TemplateAnswer.populate 1 do |other_template_answer|
other_template_answer = create_evaluation(debug, other_template_answer, call, second_template, second_auditor)
end
return template_answer
end
def create_type_of_evaluation(debug, template_answer, call, templates)
percentage_calibration = 10
percentage_double_evaluation = 50
# choose the call's user supervisor or pick random supervisors
supervisor = call.user.supervisor
# pick a random template, their number is limited
template = templates.sample
# flip coins to create calibration
template_answer = if supervisor.group.users.count > 1 && rand(100 / percentage_calibration) == 1
puts "creating calibration".colorize(:green) if debug
puts "supervisor(#{supervisor.id}) template(#{template.id})".colorize(:cyan) if debug
second_supervisor = supervisor.group.users.where("users.id not in ('#{supervisor.id}')").all.sample
puts "2nd supervisor(#{second_supervisor.id})".colorize(:cyan) if debug
create_calibration(debug, template_answer, call, template, supervisor, second_supervisor)
# end of calibration
# flip coins to create double evaluations
elsif rand(100 / percentage_double_evaluation) == 1
puts "creating double evaluations".colorize(:green) if debug
puts "supervisor(#{supervisor.id}) template(#{template.id})".colorize(:cyan) if debug
# pick random supervisors
second_supervisor = User.supervisors.where("users.id not in ('#{supervisor.id}')").all.sample
# pick a random template, their number is limited
second_template = templates.select{|t| t.id != template.id}.sample
puts "2nd supervisor(#{second_supervisor.id}) 2nd template(#{second_template.try(:id)})".colorize(:cyan) if debug
create_double_evaluations(debug, template_answer, call, template, supervisor, second_template, second_supervisor)
# end of doeble evaluations
# then create a single evaluations
else
puts "creating evaluation".colorize(:green) if debug
puts "supervisor(#{supervisor.id}) template(#{template.id})".colorize(:cyan) if debug
create_evaluation(debug, template_answer, call, template, supervisor)
end # of calibration and evaluation
end
def adjust_average_score(debug)
min_score_percentage = 40
avg_score_percentage = 80
puts "Created #{TemplateAnswer.count} evaluations".colorize(:red) if debug
puts "Adjusting the average percentage with score average to #{TemplateAnswer.average(:score_percentage)}".colorize(:green) if debug
puts "#{TemplateAnswer.where('score_percentage < 50').count} evals < 50%, #{TemplateAnswer.where('score_percentage > 50').count} evals > 50%, #{TemplateAnswer.where('score_percentage > 80').count} evals > 80%".colorize(:green) if debug
# git rid of the low score evaluations
TemplateAnswer.where("score_percentage < #{min_score_percentage}").destroy_all
# delete randomly from each percentile
iterations = (avg_score_percentage - min_score_percentage) / 10
low = min_score_percentage - 10
(1..iterations).each do |index|
low += 10
high = low + 10
evals = TemplateAnswer.where("score_percentage > #{low} and score_percentage < #{high}")
# remove heavily from low percentages
evals.limit(evals.count/10*(9-index+1)).destroy_all
end
puts "Remaining #{TemplateAnswer.count} evaluations after adjusting score average to #{TemplateAnswer.average(:score_percentage)}".colorize(:cyan) if debug
end
end # of namespace fake
end # of checking on Faker existence
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment