max_input_vars 1000 limit issue when saving theme options

[danyj]

OK I tested numerous themes with this directive being on default and either if they use ajax or not for saving theme options and they have bunch of inputs this limit has no effect. You can test redux with it ,
but Unyson theme options lazy_tabs on or off error out on save or reset if there are many options in theme settings and max_input_vars is at 1000 which is on most servers default.

error only says
] PHP Warning: Unknown: Input variables exceeded 1000. To increase the limit change max_input_vars in php.ini. in Unknown on line 0

xdebug on or off

This needs to be looked at since it is a huge issue

I think we need to chunk the theme options array when saving/resetting or ad least check this directive

if I do

fw_print ( count(fw_get_db_settings_option() ) );

I get 320 back , so yes , seems like 320 inputs ,

the size of the JSON is 34KB which is completely acceptable

avada JSON is 29KB and has about 250+ inputs

yes I have theme requirements that tells users to increase the max_input_vars but still theme options should be able to be saved/reset

[danyj]

to clarify , I dont have total of actual 320 options ,
3 options that have multiple inputs are ,

typography 5 inputs
thz_font up to 7 inputs

thz_style up to 50 inputs // this is a full CSS option that can have up to 50 but mostly has 8 or 12 , it is one option for complete CSS.

so these 3 are big but inevitable , and I cant change their inputs because of no attention to limits.

they are repeated multiple times trough out the theme option thus the big array size.

[danyj]

ha! no way , avada has 659 objects in options array and not erroring out http://prntscr.com/eqvnuz , and I have 320 http://prntscr.com/eqvo63 and are killed right a way.

please someone take a look at this, it should not be happening.

[danyj]

Redux can pack 10x more data than we can and will still not hit the limit

here is the difference, redux sends data serialized we send object

redux form data size 4

data

Unyson form data size 1165

data

[andreiglingeanu]

That is because we are treating each option type as a completely separated input and currently there's no way of changing that, unfortunately.

A proper solution for this is to send a big blob of JSON to the server and do _get_value_from_input() at the client side. @GheorgheP was digging this idea for quite some time and we actually have chatted a bit about it in Slack -- that's definitely something we are considering for the future. It is not a simple change at all, it is a fundamental change in the way Unyson is treating its options alongside with values.

[danyj]

ths is a big headache at the moment. we sent theme to testing service and this was first issue of the bet , they were not able to save/reset options so is huge fail from the start

[danyj]

@andreiglingeanu cant we just for sake of saving/resetting send serialized and catch it in between and convert to object?

[danyj]

Because option name is a must and is hard coded , https://github.com/ThemeFuse/Unyson/blob/master/framework/core/extends/class-fw-option-type.php#L158

I am not able to remove the name from fw()->backend->option_type( 'text' )->render(

in order to reduce the data in custom option type array.

I was thinking on single hidden value input with json_encode and remove names from render so that they not sent with post. this way I would reduce the form data size temporary until this is really fixed.

looks like I would have to go via regex and remove it like that for now just to test the actual idea

[danyj]

this works but major f*** is that now I have to use JS to track changes to options and add it in the single input value

form data went from 1165 to 864 by just changing one option type ,

this is huge refactoring we talking about

[andreiglingeanu]

I am not able to remove the name from fw()->backend->option_type( 'text' )->render()

I think there's no way to do this currently

I was thinking on single hidden value input with json_encode and remove names from render so that they not sent with post. this way I would reduce the form data size temporary until this is really fixed.

This is crazy

[andreiglingeanu]

Where are you facing issues mostly, Theme Settings?

[danyj]

I think there's no way to do this currently

I know

had to do this

function  _thz_remove_name_from_option($html){
	
	preg_match('/name="(.*)"/U',$html,$name);
	
	if(isset($name[0])){
		$html = str_replace($name[0],'',$html);
	}
	
	echo $html;
	
}

than in option view

			_thz_remove_name_from_option( fw()->backend->option_type( 'hidden' )->render(
				'element',
				array(
					'type'  => 'hidden',
					'value' => fw_akg('element', $option['value']),
				),
				array(
					'value' => fw_akg('element', $data['value']),
					'id_prefix' => $option['attr']['id'] .'-',
					'name_prefix' => $option['attr']['name'],
				)
			));	

[danyj]

Where are you facing issues mostly, Theme Settings?

yes

I think in versions 1.0.0 we used serialize to save data , not sure but I had this to collect form data

like

				jQuery.ajax({
					url: ajaxurl,
					type: 'POST',
					data: [
				 'action=fw_backend_options_get_values',
				 this.$form.serialize()
				  .replace(/fwf=[^\&]+\&/, ''), // remove special hidden input value to prevent form save
				 'options=' + encodeURIComponent(JSON.stringify(this.options))
				].join('&'),
					dataType: 'json',
					success: _.bind(function(response, status, xhr) {

						if (!response.success) {
							alert('Error: ' + response.data.message);
							return;
						}

						callback(response.data.values);

					}, this),
					error: function(xhr, status, error) {
						alert(status + ': ' + error.message);
					}
				});

[danyj]

so this is what I have to do now http://prntscr.com/er2t60

and than add js file that tracks changes to any of these inputs and adds that to json filed

[danyj]

for now I kinda managed it until the real issue is fixed . had to change many option types to send one input instead of multiple. I dont like it since simple inputs like select and text depend on JS now but there is no other way at the moment.

[GheorgheP]

Yes, out The Core have this problem too as it has a lot of options in Theme Settings.

I think we can try the same trick we did withe the Page Builder. To collect the all options unde one key and also compress them, then on the server to decompress.

[danyj]

under one key and also compress them, then on the server to decompress

yes , that would be the solution and is same thing others do , send one input only

[danyj]

@GheorgheP

7b1cce0

this does not seem ok

not all theme options are sent to DB on save , only the visible ones if I am not mistaking,

can you explain what exactly your process does.

[danyj]

I think this

-			afterSubmitDelay: function (elements) {		
 -				fwEvents.trigger('fw:options:init:tabs', {$elements: elements.$form});		
 -			},

and this
a621d73#diff-9cdcf26b752a9877626b45491619d439R107

was there to init all tabs and send all form data to db

and this new process seems like is sending only initiated tabs data, wich at the end when all are init has same issue , to many input vars

[danyj]

and options reset does not work right anymore, cant reset to default values via theme settings reset

[GheorgheP]

This is still under development. the idea is to send on the server only the modified values, as it doesn't make sense to send ones that wasn't modified.
And as you can see there, I set in database only the values that where submitted:

Unyson/framework/core/components/backend/class-fw-settings-form-theme.php

Line 33 in 7b1cce0

array_map('fw_set_db_settings_option', array_keys($values), $values);

[danyj]

send on the server only the modified values

That is still wrong approach and it is just hiding the actual issue which will bubble up.

If user fresh installs theme and changes 50% of options the issue will still be there.

The only way to fix this is to bundle the theme options in to one , send them to server and before they set in DB extract them via this hook
#1897

and than set them in DB

[GheorgheP]

@danyj I understand your idea, but we are talking about 2 different things. Also, in real world, did you see users that will configure all theme settings then click save?

[danyj]

l configure all theme settings then click

yes , I give them predefined theme options set ( template ) that they can click on and theme admin form gets repopulated with new options set,

like you have templates in builder I also have them for theme settings

[GheorgheP]

Ok, ok. I know question is not covering the current discussion, but, why are you populating the input fields instead just import the settings direct in database? Why do you need this step of populating the inputs so that user would then click save and then the settings to be saved?

[danyj]

instead just import the settings direct in database

I do that to with import/export option and I also have an option that saves a set of options and repopulates form with only saved set ,

for example menu settings tab has a template option to save multiple menus and have user pick the one he needs.

But no matter what I do , your approach is wrong., It is a form and on save all data should be sent to the server , we should not pick and choose what is sent to server and what should be ignored.

[danyj]

Here is prooof that reset does not work anymore

http://take.ms/edgoE

and here that your change has no effect

Original sent data before change

and here watch the video , I did not change 1 option , only clicked on tabs

http://take.ms/dLjzS

please do not do this since it is not a fix and not the right way to tackle this issue.

[GheorgheP]

@danyj What is the sens to submit the data that did not change?

Also is a wrong approach to start from form, the form is just a method to submit the data, nothing else. The real important thing is the model that renders the form, in our case the Theme Settings model. And when you update the model, then is logically to update the data that has changed, not entire model. Does this make sense for you now?

[danyj]

Does this make sense for you now

I never said it makes no sense , what I say and showed in videos above is that is not working and not fixing anything.

I am trying to make sure that we dont mess up anyone's theme settings.

[danyj]

@andreiglingeanu @ViorelEremia can you guys have a crack at this? can we somehow try to send the data as one?