Juha Yrjola <juha.yrjola@solidboot.com>
Kay Sievers <kay.sievers@vrfy.org>
Kenneth W Chen <kenneth.w.chen@intel.com>
+Konstantin Khlebnikov <koct9i@gmail.com> <k.khlebnikov@samsung.com>
Koushik <raghavendra.koushik@neterion.com>
Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Leonid I Ananiev <leonid.i.ananiev@intel.com>
event=0x2abc
event=0x423,inv,cmask=0x3
domain=0x1,offset=0x8,starting_index=0xffff
+ domain=0x1,offset=0x8,core=?
Each of the assignments indicates a value to be assigned to a
particular set of bits (as defined by the format file
corresponding to the <term>) in the perf_event structure passed
to the perf_open syscall.
+ In the case of the last example, a value replacing "?" would
+ need to be provided by the user selecting the particular event.
+ This is referred to as "event parameterization". Event
+ parameters have the format 'param=?'.
+
What: /sys/bus/event_source/devices/<pmu>/events/<event>.unit
Date: 2014/02/24
Contact: Linux kernel mailing list <linux-kernel@vger.kernel.org>
21 seconds.
This configuration parameter may be changed at runtime via the
- /sys/module/rcutree/parameters/rcu_cpu_stall_timeout, however
+ /sys/module/rcupdate/parameters/rcu_cpu_stall_timeout, however
this parameter is checked only at the beginning of a cycle.
So if you are 10 seconds into a 40-second stall, setting this
sysfs parameter to (say) five will shorten the timeout for the
"D" indicates that dyntick-idle processing is enabled ("." is printed
otherwise, for example, if disabled via the "nohz=" kernel boot parameter).
+If the relevant grace-period kthread has been unable to run prior to
+the stall warning, the following additional line is printed:
+
+ rcu_preempt kthread starved for 2023 jiffies!
+
+Starving the grace-period kthreads of CPU time can of course result in
+RCU CPU stall warnings even when all CPUs and tasks have passed through
+the required quiescent states.
+
Multiple Warnings From One Stall
behavior, you might need to replace some of the cond_resched()
calls with calls to cond_resched_rcu_qs().
+o Anything that prevents RCU's grace-period kthreads from running.
+ This can result in the "All QSes seen" console-log message.
+ This message will include information on when the kthread last
+ ran and how often it should be expected to run.
+
o A CPU-bound real-time task in a CONFIG_PREEMPT kernel, which might
happen to preempt a low-priority task in the middle of an RCU
read-side critical section. This is especially damaging if
The output of "cat rcu/rcu_preempt/rcudata" looks as follows:
- 0!c=30455 g=30456 pq=1 qp=1 dt=126535/140000000000000/0 df=2002 of=4 ql=0/0 qs=N... b=10 ci=74572 nci=0 co=1131 ca=716
- 1!c=30719 g=30720 pq=1 qp=0 dt=132007/140000000000000/0 df=1874 of=10 ql=0/0 qs=N... b=10 ci=123209 nci=0 co=685 ca=982
- 2!c=30150 g=30151 pq=1 qp=1 dt=138537/140000000000000/0 df=1707 of=8 ql=0/0 qs=N... b=10 ci=80132 nci=0 co=1328 ca=1458
- 3 c=31249 g=31250 pq=1 qp=0 dt=107255/140000000000000/0 df=1749 of=6 ql=0/450 qs=NRW. b=10 ci=151700 nci=0 co=509 ca=622
- 4!c=29502 g=29503 pq=1 qp=1 dt=83647/140000000000000/0 df=965 of=5 ql=0/0 qs=N... b=10 ci=65643 nci=0 co=1373 ca=1521
- 5 c=31201 g=31202 pq=1 qp=1 dt=70422/0/0 df=535 of=7 ql=0/0 qs=.... b=10 ci=58500 nci=0 co=764 ca=698
- 6!c=30253 g=30254 pq=1 qp=1 dt=95363/140000000000000/0 df=780 of=5 ql=0/0 qs=N... b=10 ci=100607 nci=0 co=1414 ca=1353
- 7 c=31178 g=31178 pq=1 qp=0 dt=91536/0/0 df=547 of=4 ql=0/0 qs=.... b=10 ci=109819 nci=0 co=1115 ca=969
+ 0!c=30455 g=30456 pq=1/0 qp=1 dt=126535/140000000000000/0 df=2002 of=4 ql=0/0 qs=N... b=10 ci=74572 nci=0 co=1131 ca=716
+ 1!c=30719 g=30720 pq=1/0 qp=0 dt=132007/140000000000000/0 df=1874 of=10 ql=0/0 qs=N... b=10 ci=123209 nci=0 co=685 ca=982
+ 2!c=30150 g=30151 pq=1/1 qp=1 dt=138537/140000000000000/0 df=1707 of=8 ql=0/0 qs=N... b=10 ci=80132 nci=0 co=1328 ca=1458
+ 3 c=31249 g=31250 pq=1/1 qp=0 dt=107255/140000000000000/0 df=1749 of=6 ql=0/450 qs=NRW. b=10 ci=151700 nci=0 co=509 ca=622
+ 4!c=29502 g=29503 pq=1/0 qp=1 dt=83647/140000000000000/0 df=965 of=5 ql=0/0 qs=N... b=10 ci=65643 nci=0 co=1373 ca=1521
+ 5 c=31201 g=31202 pq=1/0 qp=1 dt=70422/0/0 df=535 of=7 ql=0/0 qs=.... b=10 ci=58500 nci=0 co=764 ca=698
+ 6!c=30253 g=30254 pq=1/0 qp=1 dt=95363/140000000000000/0 df=780 of=5 ql=0/0 qs=N... b=10 ci=100607 nci=0 co=1414 ca=1353
+ 7 c=31178 g=31178 pq=1/0 qp=0 dt=91536/0/0 df=547 of=4 ql=0/0 qs=.... b=10 ci=109819 nci=0 co=1115 ca=969
This file has one line per CPU, or eight for this 8-CPU system.
The fields are as follows:
Kernels compiled with CONFIG_RCU_BOOST=y display the following from
/debug/rcu/rcu_preempt/rcudata:
- 0!c=12865 g=12866 pq=1 qp=1 dt=83113/140000000000000/0 df=288 of=11 ql=0/0 qs=N... kt=0/O ktl=944 b=10 ci=60709 nci=0 co=748 ca=871
- 1 c=14407 g=14408 pq=1 qp=0 dt=100679/140000000000000/0 df=378 of=7 ql=0/119 qs=NRW. kt=0/W ktl=9b6 b=10 ci=109740 nci=0 co=589 ca=485
- 2 c=14407 g=14408 pq=1 qp=0 dt=105486/0/0 df=90 of=9 ql=0/89 qs=NRW. kt=0/W ktl=c0c b=10 ci=83113 nci=0 co=533 ca=490
- 3 c=14407 g=14408 pq=1 qp=0 dt=107138/0/0 df=142 of=8 ql=0/188 qs=NRW. kt=0/W ktl=b96 b=10 ci=121114 nci=0 co=426 ca=290
- 4 c=14405 g=14406 pq=1 qp=1 dt=50238/0/0 df=706 of=7 ql=0/0 qs=.... kt=0/W ktl=812 b=10 ci=34929 nci=0 co=643 ca=114
- 5!c=14168 g=14169 pq=1 qp=0 dt=45465/140000000000000/0 df=161 of=11 ql=0/0 qs=N... kt=0/O ktl=b4d b=10 ci=47712 nci=0 co=677 ca=722
- 6 c=14404 g=14405 pq=1 qp=0 dt=59454/0/0 df=94 of=6 ql=0/0 qs=.... kt=0/W ktl=e57 b=10 ci=55597 nci=0 co=701 ca=811
- 7 c=14407 g=14408 pq=1 qp=1 dt=68850/0/0 df=31 of=8 ql=0/0 qs=.... kt=0/W ktl=14bd b=10 ci=77475 nci=0 co=508 ca=1042
+ 0!c=12865 g=12866 pq=1/0 qp=1 dt=83113/140000000000000/0 df=288 of=11 ql=0/0 qs=N... kt=0/O ktl=944 b=10 ci=60709 nci=0 co=748 ca=871
+ 1 c=14407 g=14408 pq=1/0 qp=0 dt=100679/140000000000000/0 df=378 of=7 ql=0/119 qs=NRW. kt=0/W ktl=9b6 b=10 ci=109740 nci=0 co=589 ca=485
+ 2 c=14407 g=14408 pq=1/0 qp=0 dt=105486/0/0 df=90 of=9 ql=0/89 qs=NRW. kt=0/W ktl=c0c b=10 ci=83113 nci=0 co=533 ca=490
+ 3 c=14407 g=14408 pq=1/0 qp=0 dt=107138/0/0 df=142 of=8 ql=0/188 qs=NRW. kt=0/W ktl=b96 b=10 ci=121114 nci=0 co=426 ca=290
+ 4 c=14405 g=14406 pq=1/0 qp=1 dt=50238/0/0 df=706 of=7 ql=0/0 qs=.... kt=0/W ktl=812 b=10 ci=34929 nci=0 co=643 ca=114
+ 5!c=14168 g=14169 pq=1/0 qp=0 dt=45465/140000000000000/0 df=161 of=11 ql=0/0 qs=N... kt=0/O ktl=b4d b=10 ci=47712 nci=0 co=677 ca=722
+ 6 c=14404 g=14405 pq=1/0 qp=0 dt=59454/0/0 df=94 of=6 ql=0/0 qs=.... kt=0/W ktl=e57 b=10 ci=55597 nci=0 co=701 ca=811
+ 7 c=14407 g=14408 pq=1/0 qp=1 dt=68850/0/0 df=31 of=8 ql=0/0 qs=.... kt=0/W ktl=14bd b=10 ci=77475 nci=0 co=508 ca=1042
This is similar to the output discussed above, but contains the following
additional fields:
compatible = "st,comms-ssc4-i2c";
reg = <0xfed40000 0x110>;
interrupts = <GIC_SPI 187 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&CLK_S_ICN_REG_0>;
+ clocks = <&clk_s_a0_ls CLK_ICN_REG>;
clock-names = "ssc";
clock-frequency = <400000>;
pinctrl-names = "default";
dallas,ds4510 CPU Supervisor with Nonvolatile Memory and Programmable I/O
dallas,ds75 Digital Thermometer and Thermostat
dlg,da9053 DA9053: flexible system level PMIC with multicore support
+dlg,da9063 DA9063: system PMIC for quad-core application processors
epson,rx8025 High-Stability. I2C-Bus INTERFACE REAL TIME CLOCK MODULE
epson,rx8581 I2C-BUS INTERFACE REAL TIME CLOCK MODULE
fsl,mag3110 MAG3110: Xtrinsic High Accuracy, 3D Magnetometer
-BUCKn : 1-4.
Use standard regulator bindings for it ('regulator-off-in-suspend').
+ LDO20, LDO21, LDO22, BUCK8 and BUCK9 can be configured to GPIO enable
+ control. To turn this feature on this property must be added to the regulator
+ sub-node:
+ - maxim,ena-gpios : one GPIO specifier enable control (the gpio
+ flags are actually ignored and always
+ ACTIVE_HIGH is used)
Example:
regulator-always-on;
regulator-boot-on;
};
+
+ buck9_reg {
+ regulator-compatible = "BUCK9";
+ regulator-name = "CAM_ISP_CORE_1.2V";
+ regulator-min-microvolt = <1000000>;
+ regulator-max-microvolt = <1200000>;
+ maxim,ena-gpios = <&gpm0 3 GPIO_ACTIVE_HIGH>;
+ };
}
BUCKA and BUCKB.
Optional properties:
+- enable-gpios: platform gpio for control of BUCKA/BUCKB.
- Any optional property defined in regulator.txt
Example 1) DA9211
regulator-max-microvolt = <1570000>;
regulator-min-microamp = <2000000>;
regulator-max-microamp = <5000000>;
+ enable-gpios = <&gpio 27 0>;
};
BUCKB {
regulator-name = "VBUCKB";
regulator-max-microvolt = <1570000>;
regulator-min-microamp = <2000000>;
regulator-max-microamp = <5000000>;
+ enable-gpios = <&gpio 17 0>;
};
};
};
-Example 2) DA92113
+Example 2) DA9213
pmic: da9213@68 {
compatible = "dlg,da9213";
reg = <0x68>;
regulator-max-microvolt = <1570000>;
regulator-min-microamp = <3000000>;
regulator-max-microamp = <6000000>;
+ enable-gpios = <&gpio 27 0>;
};
BUCKB {
regulator-name = "VBUCKB";
regulator-max-microvolt = <1570000>;
regulator-min-microamp = <3000000>;
regulator-max-microamp = <6000000>;
+ enable-gpios = <&gpio 17 0>;
};
};
};
Required properties:
-- compatible: "isl,isl9305" or "isl,isl9305h"
+- compatible: "isil,isl9305" or "isil,isl9305h"
- reg: I2C slave address, usually 0x68.
- regulators: A node that houses a sub-node for each regulator within the
device. Each sub-node is identified using the node's name, with valid
Example
pmic: isl9305@68 {
- compatible = "isl,isl9305";
+ compatible = "isil,isl9305";
reg = <0x68>;
VINDCD1-supply = <&system_power>;
--- /dev/null
+Mediatek MT6397 Regulator Driver
+
+Required properties:
+- compatible: "mediatek,mt6397-regulator"
+- mt6397regulator: List of regulators provided by this controller. It is named
+ according to its regulator type, buck_<name> and ldo_<name>.
+ The definition for each of these nodes is defined using the standard binding
+ for regulators at Documentation/devicetree/bindings/regulator/regulator.txt.
+
+The valid names for regulators are::
+BUCK:
+ buck_vpca15, buck_vpca7, buck_vsramca15, buck_vsramca7, buck_vcore, buck_vgpu,
+ buck_vdrm, buck_vio18
+LDO:
+ ldo_vtcxo, ldo_va28, ldo_vcama, ldo_vio28, ldo_vusb, ldo_vmc, ldo_vmch,
+ ldo_vemc3v3, ldo_vgp1, ldo_vgp2, ldo_vgp3, ldo_vgp4, ldo_vgp5, ldo_vgp6,
+ ldo_vibr
+
+Example:
+ pmic {
+ compatible = "mediatek,mt6397";
+
+ mt6397regulator: mt6397regulator {
+ compatible = "mediatek,mt6397-regulator";
+
+ mt6397_vpca15_reg: buck_vpca15 {
+ regulator-compatible = "buck_vpca15";
+ regulator-name = "vpca15";
+ regulator-min-microvolt = < 850000>;
+ regulator-max-microvolt = <1350000>;
+ regulator-ramp-delay = <12500>;
+ regulator-enable-ramp-delay = <200>;
+ };
+
+ mt6397_vpca7_reg: buck_vpca7 {
+ regulator-compatible = "buck_vpca7";
+ regulator-name = "vpca7";
+ regulator-min-microvolt = < 850000>;
+ regulator-max-microvolt = <1350000>;
+ regulator-ramp-delay = <12500>;
+ regulator-enable-ramp-delay = <115>;
+ };
+
+ mt6397_vsramca15_reg: buck_vsramca15 {
+ regulator-compatible = "buck_vsramca15";
+ regulator-name = "vsramca15";
+ regulator-min-microvolt = < 850000>;
+ regulator-max-microvolt = <1350000>;
+ regulator-ramp-delay = <12500>;
+ regulator-enable-ramp-delay = <115>;
+
+ };
+
+ mt6397_vsramca7_reg: buck_vsramca7 {
+ regulator-compatible = "buck_vsramca7";
+ regulator-name = "vsramca7";
+ regulator-min-microvolt = < 850000>;
+ regulator-max-microvolt = <1350000>;
+ regulator-ramp-delay = <12500>;
+ regulator-enable-ramp-delay = <115>;
+
+ };
+
+ mt6397_vcore_reg: buck_vcore {
+ regulator-compatible = "buck_vcore";
+ regulator-name = "vcore";
+ regulator-min-microvolt = < 850000>;
+ regulator-max-microvolt = <1350000>;
+ regulator-ramp-delay = <12500>;
+ regulator-enable-ramp-delay = <115>;
+ };
+
+ mt6397_vgpu_reg: buck_vgpu {
+ regulator-compatible = "buck_vgpu";
+ regulator-name = "vgpu";
+ regulator-min-microvolt = < 700000>;
+ regulator-max-microvolt = <1350000>;
+ regulator-ramp-delay = <12500>;
+ regulator-enable-ramp-delay = <115>;
+ };
+
+ mt6397_vdrm_reg: buck_vdrm {
+ regulator-compatible = "buck_vdrm";
+ regulator-name = "vdrm";
+ regulator-min-microvolt = < 800000>;
+ regulator-max-microvolt = <1400000>;
+ regulator-ramp-delay = <12500>;
+ regulator-enable-ramp-delay = <500>;
+ };
+
+ mt6397_vio18_reg: buck_vio18 {
+ regulator-compatible = "buck_vio18";
+ regulator-name = "vio18";
+ regulator-min-microvolt = <1500000>;
+ regulator-max-microvolt = <2120000>;
+ regulator-ramp-delay = <12500>;
+ regulator-enable-ramp-delay = <500>;
+ };
+
+ mt6397_vtcxo_reg: ldo_vtcxo {
+ regulator-compatible = "ldo_vtcxo";
+ regulator-name = "vtcxo";
+ regulator-min-microvolt = <2800000>;
+ regulator-max-microvolt = <2800000>;
+ regulator-enable-ramp-delay = <90>;
+ };
+
+ mt6397_va28_reg: ldo_va28 {
+ regulator-compatible = "ldo_va28";
+ regulator-name = "va28";
+ /* fixed output 2.8 V */
+ regulator-enable-ramp-delay = <218>;
+ };
+
+ mt6397_vcama_reg: ldo_vcama {
+ regulator-compatible = "ldo_vcama";
+ regulator-name = "vcama";
+ regulator-min-microvolt = <1500000>;
+ regulator-max-microvolt = <2800000>;
+ regulator-enable-ramp-delay = <218>;
+ };
+
+ mt6397_vio28_reg: ldo_vio28 {
+ regulator-compatible = "ldo_vio28";
+ regulator-name = "vio28";
+ /* fixed output 2.8 V */
+ regulator-enable-ramp-delay = <240>;
+ };
+
+ mt6397_usb_reg: ldo_vusb {
+ regulator-compatible = "ldo_vusb";
+ regulator-name = "vusb";
+ /* fixed output 3.3 V */
+ regulator-enable-ramp-delay = <218>;
+ };
+
+ mt6397_vmc_reg: ldo_vmc {
+ regulator-compatible = "ldo_vmc";
+ regulator-name = "vmc";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-enable-ramp-delay = <218>;
+ };
+
+ mt6397_vmch_reg: ldo_vmch {
+ regulator-compatible = "ldo_vmch";
+ regulator-name = "vmch";
+ regulator-min-microvolt = <3000000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-enable-ramp-delay = <218>;
+ };
+
+ mt6397_vemc_3v3_reg: ldo_vemc3v3 {
+ regulator-compatible = "ldo_vemc3v3";
+ regulator-name = "vemc_3v3";
+ regulator-min-microvolt = <3000000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-enable-ramp-delay = <218>;
+ };
+
+ mt6397_vgp1_reg: ldo_vgp1 {
+ regulator-compatible = "ldo_vgp1";
+ regulator-name = "vcamd";
+ regulator-min-microvolt = <1220000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-enable-ramp-delay = <240>;
+ };
+
+ mt6397_vgp2_reg: ldo_vgp2 {
+ egulator-compatible = "ldo_vgp2";
+ regulator-name = "vcamio";
+ regulator-min-microvolt = <1000000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-enable-ramp-delay = <218>;
+ };
+
+ mt6397_vgp3_reg: ldo_vgp3 {
+ regulator-compatible = "ldo_vgp3";
+ regulator-name = "vcamaf";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-enable-ramp-delay = <218>;
+ };
+
+ mt6397_vgp4_reg: ldo_vgp4 {
+ regulator-compatible = "ldo_vgp4";
+ regulator-name = "vgp4";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-enable-ramp-delay = <218>;
+ };
+
+ mt6397_vgp5_reg: ldo_vgp5 {
+ regulator-compatible = "ldo_vgp5";
+ regulator-name = "vgp5";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <3000000>;
+ regulator-enable-ramp-delay = <218>;
+ };
+
+ mt6397_vgp6_reg: ldo_vgp6 {
+ regulator-compatible = "ldo_vgp6";
+ regulator-name = "vgp6";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-enable-ramp-delay = <218>;
+ };
+
+ mt6397_vibr_reg: ldo_vibr {
+ regulator-compatible = "ldo_vibr";
+ regulator-name = "vibr";
+ regulator-min-microvolt = <1200000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-enable-ramp-delay = <218>;
+ };
+ };
+ };
PFUZE100 family of regulators
Required properties:
-- compatible: "fsl,pfuze100" or "fsl,pfuze200"
+- compatible: "fsl,pfuze100", "fsl,pfuze200", "fsl,pfuze3000"
- reg: I2C slave address
Required child node:
sw1ab,sw1c,sw2,sw3a,sw3b,sw4,swbst,vsnvs,vrefddr,vgen1~vgen6
--PFUZE200
sw1ab,sw2,sw3a,sw3b,swbst,vsnvs,vrefddr,vgen1~vgen6
+ --PFUZE3000
+ sw1a,sw1b,sw2,sw3,swbst,vsnvs,vrefddr,vldo1,vldo2,vccsd,v33,vldo3,vldo4
Each regulator is defined using the standard binding for regulators.
};
};
};
+
+Example 3: PFUZE3000
+
+ pmic: pfuze3000@08 {
+ compatible = "fsl,pfuze3000";
+ reg = <0x08>;
+
+ regulators {
+ sw1a_reg: sw1a {
+ regulator-min-microvolt = <700000>;
+ regulator-max-microvolt = <1475000>;
+ regulator-boot-on;
+ regulator-always-on;
+ regulator-ramp-delay = <6250>;
+ };
+ /* use sw1c_reg to align with pfuze100/pfuze200 */
+ sw1c_reg: sw1b {
+ regulator-min-microvolt = <700000>;
+ regulator-max-microvolt = <1475000>;
+ regulator-boot-on;
+ regulator-always-on;
+ regulator-ramp-delay = <6250>;
+ };
+
+ sw2_reg: sw2 {
+ regulator-min-microvolt = <2500000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ sw3a_reg: sw3 {
+ regulator-min-microvolt = <900000>;
+ regulator-max-microvolt = <1650000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ swbst_reg: swbst {
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5150000>;
+ };
+
+ snvs_reg: vsnvs {
+ regulator-min-microvolt = <1000000>;
+ regulator-max-microvolt = <3000000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ vref_reg: vrefddr {
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ vgen1_reg: vldo1 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ };
+
+ vgen2_reg: vldo2 {
+ regulator-min-microvolt = <800000>;
+ regulator-max-microvolt = <1550000>;
+ };
+
+ vgen3_reg: vccsd {
+ regulator-min-microvolt = <2850000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ };
+
+ vgen4_reg: v33 {
+ regulator-min-microvolt = <2850000>;
+ regulator-max-microvolt = <3300000>;
+ };
+
+ vgen5_reg: vldo3 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ };
+
+ vgen6_reg: vldo4 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ };
+ };
+ };
specifiers, one for transmission, and one for
reception.
- dma-names : Must contain a list of two DMA names, "tx" and "rx".
+- renesas,dtdl : delay sync signal (setup) in transmit mode.
+ Must contain one of the following values:
+ 0 (no bit delay)
+ 50 (0.5-clock-cycle delay)
+ 100 (1-clock-cycle delay)
+ 150 (1.5-clock-cycle delay)
+ 200 (2-clock-cycle delay)
+
+- renesas,syncdl : delay sync signal (hold) in transmit mode.
+ Must contain one of the following values:
+ 0 (no bit delay)
+ 50 (0.5-clock-cycle delay)
+ 100 (1-clock-cycle delay)
+ 150 (1.5-clock-cycle delay)
+ 200 (2-clock-cycle delay)
+ 300 (3-clock-cycle delay)
Optional properties, deprecated for soctype-specific bindings:
- renesas,tx-fifo-size : Overrides the default tx fifo size given in words
--- /dev/null
+* CSR SiRFprimaII Serial Peripheral Interface
+
+Required properties:
+- compatible : Should be "sirf,prima2-spi"
+- reg : Offset and length of the register set for the device
+- interrupts : Should contain SPI interrupt
+- resets: phandle to the reset controller asserting this device in
+ reset
+ See ../reset/reset.txt for details.
+- dmas : Must contain an entry for each entry in clock-names.
+ See ../dma/dma.txt for details.
+- dma-names : Must include the following entries:
+ - rx
+ - tx
+- clocks : Must contain an entry for each entry in clock-names.
+ See ../clocks/clock-bindings.txt for details.
+
+- #address-cells: Number of cells required to define a chip select
+ address on the SPI bus. Should be set to 1.
+- #size-cells: Should be zero.
+
+Optional properties:
+- spi-max-frequency: Specifies maximum SPI clock frequency,
+ Units - Hz. Definition as per
+ Documentation/devicetree/bindings/spi/spi-bus.txt
+- cs-gpios: should specify GPIOs used for chipselects.
+
+Example:
+
+spi0: spi@b00d0000 {
+ compatible = "sirf,prima2-spi";
+ reg = <0xb00d0000 0x10000>;
+ interrupts = <15>;
+ dmas = <&dmac1 9>,
+ <&dmac1 4>;
+ dma-names = "rx", "tx";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ clocks = <&clks 19>;
+ resets = <&rstc 26>;
+};
--- /dev/null
+STMicroelectronics SSC (SPI) Controller
+---------------------------------------
+
+Required properties:
+- compatible : "st,comms-ssc4-spi"
+- reg : Offset and length of the device's register set
+- interrupts : The interrupt specifier
+- clock-names : Must contain "ssc"
+- clocks : Must contain an entry for each name in clock-names
+ See ../clk/*
+- pinctrl-names : Uses "default", can use "sleep" if provided
+ See ../pinctrl/pinctrl-binding.txt
+
+Optional properties:
+- cs-gpios : List of GPIO chip selects
+ See ../spi/spi-bus.txt
+
+Child nodes represent devices on the SPI bus
+ See ../spi/spi-bus.txt
+
+Example:
+ spi@9840000 {
+ compatible = "st,comms-ssc4-spi";
+ reg = <0x9840000 0x110>;
+ interrupts = <GIC_SPI 112 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&clk_s_c0_flexgen CLK_EXT2F_A9>;
+ clock-names = "ssc";
+ pinctrl-0 = <&pinctrl_spi0_default>;
+ pinctrl-names = "default";
+ cs-gpios = <&pio17 5 0>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ st95hf@0{
+ compatible = "st,st95hf";
+ reg = <0>;
+ spi-max-frequency = <1000000>;
+ interrupts = <2 IRQ_TYPE_EDGE_FALLING>;
+ };
+ };
allow the requeue code to acquire an uncontended rt_mutex on behalf
of the waiter and to enqueue the waiter on a contended rt_mutex.
Two new system calls provide the kernel<->user interface to
-requeue_pi: FUTEX_WAIT_REQUEUE_PI and FUTEX_REQUEUE_CMP_PI.
+requeue_pi: FUTEX_WAIT_REQUEUE_PI and FUTEX_CMP_REQUEUE_PI.
FUTEX_WAIT_REQUEUE_PI is called by the waiter (pthread_cond_wait()
and pthread_cond_timedwait()) to block on the initial futex and wait
futex_lock_pi(), with some extra logic to check for the additional
wake-up scenarios.
-FUTEX_REQUEUE_CMP_PI is called by the waker
+FUTEX_CMP_REQUEUE_PI is called by the waker
(pthread_cond_broadcast() and pthread_cond_signal()) to requeue and
possibly wake the waiting tasks. Internally, this system call is
still handled by futex_requeue (by passing requeue_pi=1). Before
the lock can be acquired at this stage as well, if so, the next
waiter is woken to finish the acquisition of the lock.
-FUTEX_REQUEUE_PI accepts nr_wake and nr_requeue as arguments, but
+FUTEX_CMP_REQUEUE_PI accepts nr_wake and nr_requeue as arguments, but
their sum is all that really matters. futex_requeue() will wake or
requeue up to nr_wake + nr_requeue tasks. It will wake only as many
tasks as it can acquire the lock for, which in the majority of cases
should be 0 as good programming practice dictates that the caller of
either pthread_cond_broadcast() or pthread_cond_signal() acquire the
-mutex prior to making the call. FUTEX_REQUEUE_PI requires that
+mutex prior to making the call. FUTEX_CMP_REQUEUE_PI requires that
nr_wake=1. nr_requeue should be INT_MAX for broadcast and 0 for
signal.
Datasheet: Publicly available at the Texas Instruments website
http://www.ti.com/
+ * Texas Instruments INA231
+ Prefix: 'ina231'
+ Addresses: I2C 0x40 - 0x4f
+ Datasheet: Publicly available at the Texas Instruments website
+ http://www.ti.com/
+
Author: Lothar Felten <l-felten@ti.com>
Description
The INA226 is a current shunt and power monitor with an I2C interface.
The INA226 monitors both a shunt voltage drop and bus supply voltage.
-The INA230 is a high or low side current shunt and power monitor with an I2C
-interface. The INA230 monitors both a shunt voltage drop and bus supply voltage.
+INA230 and INA231 are high or low side current shunt and power monitors
+with an I2C interface. The chips monitor both a shunt voltage drop and
+bus supply voltage.
-The shunt value in micro-ohms can be set via platform data or device tree.
-Please refer to the Documentation/devicetree/bindings/i2c/ina2xx.txt for bindings
+The shunt value in micro-ohms can be set via platform data or device tree at
+compile-time or via the shunt_resistor attribute in sysfs at run-time. Please
+refer to the Documentation/devicetree/bindings/i2c/ina2xx.txt for bindings
if the device tree is used.
+
+Additionally ina226 supports update_interval attribute as described in
+Documentation/hwmon/sysfs-interface. Internally the interval is the sum of
+bus and shunt voltage conversion times multiplied by the averaging rate. We
+don't touch the conversion times and only modify the number of averages. The
+lower limit of the update_interval is 2 ms, the upper limit is 2253 ms.
+The actual programmed interval may vary from the desired value.
- 'ever held with STATE enabled'
- 'ever held as readlock with STATE enabled'
-Where STATE can be either one of (kernel/lockdep_states.h)
+Where STATE can be either one of (kernel/locking/lockdep_states.h)
- hardirq
- softirq
- reclaim_fs
STORE *(A + 4) = Y; STORE *A = X;
STORE {*A, *(A + 4) } = {X, Y};
+And there are anti-guarantees:
+
+ (*) These guarantees do not apply to bitfields, because compilers often
+ generate code to modify these using non-atomic read-modify-write
+ sequences. Do not attempt to use bitfields to synchronize parallel
+ algorithms.
+
+ (*) Even in cases where bitfields are protected by locks, all fields
+ in a given bitfield must be protected by one lock. If two fields
+ in a given bitfield are protected by different locks, the compiler's
+ non-atomic read-modify-write sequences can cause an update to one
+ field to corrupt the value of an adjacent field.
+
+ (*) These guarantees apply only to properly aligned and sized scalar
+ variables. "Properly sized" currently means variables that are
+ the same size as "char", "short", "int" and "long". "Properly
+ aligned" means the natural alignment, thus no constraints for
+ "char", two-byte alignment for "short", four-byte alignment for
+ "int", and either four-byte or eight-byte alignment for "long",
+ on 32-bit and 64-bit systems, respectively. Note that these
+ guarantees were introduced into the C11 standard, so beware when
+ using older pre-C11 compilers (for example, gcc 4.6). The portion
+ of the standard containing this guarantee is Section 3.14, which
+ defines "memory location" as follows:
+
+ memory location
+ either an object of scalar type, or a maximal sequence
+ of adjacent bit-fields all having nonzero width
+
+ NOTE 1: Two threads of execution can update and access
+ separate memory locations without interfering with
+ each other.
+
+ NOTE 2: A bit-field and an adjacent non-bit-field member
+ are in separate memory locations. The same applies
+ to two bit-fields, if one is declared inside a nested
+ structure declaration and the other is not, or if the two
+ are separated by a zero-length bit-field declaration,
+ or if they are separated by a non-bit-field member
+ declaration. It is not safe to concurrently update two
+ bit-fields in the same structure if all members declared
+ between them are also bit-fields, no matter what the
+ sizes of those intervening bit-fields happen to be.
+
=========================
WHAT ARE MEMORY BARRIERS?
However, they do -not- guarantee any other sort of ordering:
Not prior loads against later loads, nor prior stores against
later anything. If you need these other forms of ordering,
- use smb_rmb(), smp_wmb(), or, in the case of prior stores and
+ use smp_rmb(), smp_wmb(), or, in the case of prior stores and
later loads, smp_mb().
(*) If both legs of the "if" statement begin with identical stores
TX limitations
--------------
-Kernel processing usually involves validation of the message received by
-user-space, then processing its contents. The kernel must assure that
-userspace is not able to modify the message contents after they have been
-validated. In order to do so, the message is copied from the ring frame
-to an allocated buffer if either of these conditions is false:
-
-- only a single mapping of the ring exists
-- the file descriptor is not shared between processes
-
-This means that for threaded programs, the kernel will fall back to copying.
+As of Jan 2015 the message is always copied from the ring frame to an
+allocated buffer due to unresolved security concerns.
+See commit 4682a0358639b29cf ("netlink: Always copy on mmap TX.").
Example
-------
xorl %ebx,%ebx
1: ret
-and the whole paranoid non-paranoid macro complexity is about whether
-to suffer that RDMSR cost.
-
If we are at an interrupt or user-trap/gate-alike boundary then we can
use the faster check: the stack will be a reliable indicator of
whether SWAPGS was already done: if we see that we are a secondary
stack but before we executed SWAPGS, then the only safe way to check
for GS is the slower method: the RDMSR.
-So we try only to mark those entry methods 'paranoid' that absolutely
-need the more expensive check for the GS base - and we generate all
-'normal' entry points with the regular (faster) entry macros.
+Therefore, super-atomic entries (except NMI, which is handled separately)
+must use idtentry with paranoid=1 to handle gsbase correctly. This
+triggers three main behavior changes:
+
+ - Interrupt entry will use the slower gsbase check.
+ - Interrupt entry from user mode will switch off the IST stack.
+ - Interrupt exit to kernel mode will not attempt to reschedule.
+
+We try to only use IST entries and the paranoid entry code for vectors
+that absolutely need the more expensive check for the GS base - and we
+generate all 'normal' entry points with the regular (faster) paranoid=0
+variant.
interrupt-gate descriptor. When an interrupt occurs and the hardware
loads such a descriptor, the hardware automatically sets the new stack
pointer based on the IST value, then invokes the interrupt handler. If
-software wants to allow nested IST interrupts then the handler must
-adjust the IST values on entry to and exit from the interrupt handler.
-(This is occasionally done, e.g. for debug exceptions.)
+the interrupt came from user mode, then the interrupt handler prologue
+will switch back to the per-thread stack. If software wants to allow
+nested IST interrupts then the handler must adjust the IST values on
+entry to and exit from the interrupt handler. (This is occasionally
+done, e.g. for debug exceptions.)
Events with different IST codes (i.e. with different stacks) can be
nested. For example, a debug interrupt can safely be interrupted by an
F: drivers/staging/iio/*/ad*
F: staging/iio/trigger/iio-trig-bfin-timer.c
+ANDROID DRIVERS
+M: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
+M: Arve Hjønnevåg <arve@android.com>
+M: Riley Andrews <riandrews@android.com>
+T: git git://git.kernel.org/pub/scm/linux/kernel/gregkh/staging.git
+L: devel@driverdev.osuosl.org
+S: Supported
+F: drivers/android/
+F: drivers/staging/android/
+
AOA (Apple Onboard Audio) ALSA DRIVER
M: Johannes Berg <johannes@sipsolutions.net>
L: linuxppc-dev@lists.ozlabs.org
F: drivers/input/input-mt.c
K: \b(ABS|SYN)_MT_
+INTEL ASoC BDW/HSW DRIVERS
+M: Jie Yang <yang.jie@linux.intel.com>
+L: alsa-devel@alsa-project.org
+S: Supported
+F: sound/soc/intel/sst-haswell*
+F: sound/soc/intel/sst-dsp*
+F: sound/soc/intel/sst-firmware.c
+F: sound/soc/intel/broadwell.c
+F: sound/soc/intel/haswell.c
+
INTEL C600 SERIES SAS CONTROLLER DRIVER
M: Intel SCU Linux support <intel-linux-scu@intel.com>
M: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
SUPERH
L: linux-sh@vger.kernel.org
-W: http://www.linux-sh.org
Q: http://patchwork.kernel.org/project/linux-sh/list/
S: Orphan
F: Documentation/sh/
M: "Hans J. Koch" <hjk@hansjkoch.de>
M: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
S: Maintained
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git
F: Documentation/DocBook/uio-howto.tmpl
F: drivers/uio/
F: include/linux/uio*.h
VERSION = 3
PATCHLEVEL = 19
SUBLEVEL = 0
-EXTRAVERSION = -rc6
+EXTRAVERSION =
NAME = Diseased Newt
# *DOCUMENTATION*
if (unlikely(fault & VM_FAULT_ERROR)) {
if (fault & VM_FAULT_OOM)
goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
else if (fault & VM_FAULT_SIGBUS)
goto do_sigbus;
BUG();
if (fault & VM_FAULT_OOM)
goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
else if (fault & VM_FAULT_SIGBUS)
goto do_sigbus;
* OK... Let's do some funky business here.
* If we do have a DTB appended to zImage, and we do have
* an ATAG list around, we want the later to be translated
- * and folded into the former here. To be on the safe side,
- * let's temporarily move the stack away into the malloc
- * area. No GOT fixup has occurred yet, but none of the
- * code we're about to call uses any global variable.
+ * and folded into the former here. No GOT fixup has occurred
+ * yet, but none of the code we're about to call uses any
+ * global variable.
*/
- add sp, sp, #0x10000
+
+ /* Get the initial DTB size */
+ ldr r5, [r6, #4]
+#ifndef __ARMEB__
+ /* convert to little endian */
+ eor r1, r5, r5, ror #16
+ bic r1, r1, #0x00ff0000
+ mov r5, r5, ror #8
+ eor r5, r5, r1, lsr #8
+#endif
+ /* 50% DTB growth should be good enough */
+ add r5, r5, r5, lsr #1
+ /* preserve 64-bit alignment */
+ add r5, r5, #7
+ bic r5, r5, #7
+ /* clamp to 32KB min and 1MB max */
+ cmp r5, #(1 << 15)
+ movlo r5, #(1 << 15)
+ cmp r5, #(1 << 20)
+ movhi r5, #(1 << 20)
+ /* temporarily relocate the stack past the DTB work space */
+ add sp, sp, r5
+
stmfd sp!, {r0-r3, ip, lr}
mov r0, r8
mov r1, r6
- sub r2, sp, r6
+ mov r2, r5
bl atags_to_fdt
/*
bic r0, r0, #1
add r0, r0, #0x100
mov r1, r6
- sub r2, sp, r6
+ mov r2, r5
bleq atags_to_fdt
ldmfd sp!, {r0-r3, ip, lr}
- sub sp, sp, #0x10000
+ sub sp, sp, r5
#endif
mov r8, r6 @ use the appended device tree
subs r1, r5, r1
addhi r9, r9, r1
- /* Get the dtb's size */
+ /* Get the current DTB size */
ldr r5, [r6, #4]
#ifndef __ARMEB__
/* convert r5 (dtb size) to little endian */
};
i2s1: i2s@13960000 {
- compatible = "samsung,s5pv210-i2s";
+ compatible = "samsung,s3c6410-i2s";
reg = <0x13960000 0x100>;
clocks = <&clock CLK_I2S1>;
clock-names = "iis";
};
i2s2: i2s@13970000 {
- compatible = "samsung,s5pv210-i2s";
+ compatible = "samsung,s3c6410-i2s";
reg = <0x13970000 0x100>;
clocks = <&clock CLK_I2S2>;
clock-names = "iis";
aliases {
ethernet0 = &emac;
- serial0 = &uart0;
- serial1 = &uart1;
- serial2 = &uart2;
- serial3 = &uart3;
- serial4 = &uart4;
- serial5 = &uart5;
- serial6 = &uart6;
- serial7 = &uart7;
};
chosen {
<&ahb_gates 44>;
status = "disabled";
};
+
+ framebuffer@1 {
+ compatible = "allwinner,simple-framebuffer", "simple-framebuffer";
+ allwinner,pipeline = "de_fe0-de_be0-lcd0-hdmi";
+ clocks = <&pll5 1>, <&ahb_gates 36>, <&ahb_gates 43>,
+ <&ahb_gates 44>, <&ahb_gates 46>;
+ status = "disabled";
+ };
};
cpus {
reg-names = "phy_ctrl", "pmu1", "pmu2";
clocks = <&usb_clk 8>;
clock-names = "usb_phy";
- resets = <&usb_clk 1>, <&usb_clk 2>;
- reset-names = "usb1_reset", "usb2_reset";
+ resets = <&usb_clk 0>, <&usb_clk 1>, <&usb_clk 2>;
+ reset-names = "usb0_reset", "usb1_reset", "usb2_reset";
status = "disabled";
};
model = "Olimex A10s-Olinuxino Micro";
compatible = "olimex,a10s-olinuxino-micro", "allwinner,sun5i-a10s";
+ aliases {
+ serial0 = &uart0;
+ serial1 = &uart2;
+ serial2 = &uart3;
+ };
+
soc@01c00000 {
emac: ethernet@01c0b000 {
pinctrl-names = "default";
aliases {
ethernet0 = &emac;
- serial0 = &uart0;
- serial1 = &uart1;
- serial2 = &uart2;
- serial3 = &uart3;
};
chosen {
reg-names = "phy_ctrl", "pmu1";
clocks = <&usb_clk 8>;
clock-names = "usb_phy";
- resets = <&usb_clk 1>;
- reset-names = "usb1_reset";
+ resets = <&usb_clk 0>, <&usb_clk 1>;
+ reset-names = "usb0_reset", "usb1_reset";
status = "disabled";
};
model = "HSG H702";
compatible = "hsg,h702", "allwinner,sun5i-a13";
+ aliases {
+ serial0 = &uart1;
+ };
+
soc@01c00000 {
mmc0: mmc@01c0f000 {
pinctrl-names = "default";
model = "Olimex A13-Olinuxino Micro";
compatible = "olimex,a13-olinuxino-micro", "allwinner,sun5i-a13";
+ aliases {
+ serial0 = &uart1;
+ };
+
soc@01c00000 {
mmc0: mmc@01c0f000 {
pinctrl-names = "default";
model = "Olimex A13-Olinuxino";
compatible = "olimex,a13-olinuxino", "allwinner,sun5i-a13";
+ aliases {
+ serial0 = &uart1;
+ };
+
soc@01c00000 {
mmc0: mmc@01c0f000 {
pinctrl-names = "default";
/ {
interrupt-parent = <&intc>;
- aliases {
- serial0 = &uart1;
- serial1 = &uart3;
- };
-
cpus {
#address-cells = <1>;
#size-cells = <0>;
reg-names = "phy_ctrl", "pmu1";
clocks = <&usb_clk 8>;
clock-names = "usb_phy";
- resets = <&usb_clk 1>;
- reset-names = "usb1_reset";
+ resets = <&usb_clk 0>, <&usb_clk 1>;
+ reset-names = "usb0_reset", "usb1_reset";
status = "disabled";
};
interrupt-parent = <&gic>;
aliases {
- serial0 = &uart0;
- serial1 = &uart1;
- serial2 = &uart2;
- serial3 = &uart3;
- serial4 = &uart4;
- serial5 = &uart5;
ethernet0 = &gmac;
};
model = "LeMaker Banana Pi";
compatible = "lemaker,bananapi", "allwinner,sun7i-a20";
+ aliases {
+ serial0 = &uart0;
+ serial1 = &uart3;
+ serial2 = &uart7;
+ };
+
soc@01c00000 {
spi0: spi@01c05000 {
pinctrl-names = "default";
model = "Merrii A20 Hummingbird";
compatible = "merrii,a20-hummingbird", "allwinner,sun7i-a20";
+ aliases {
+ serial0 = &uart0;
+ serial1 = &uart2;
+ serial2 = &uart3;
+ serial3 = &uart4;
+ serial4 = &uart5;
+ };
+
soc@01c00000 {
mmc0: mmc@01c0f000 {
pinctrl-names = "default";
compatible = "olimex,a20-olinuxino-micro", "allwinner,sun7i-a20";
aliases {
+ serial0 = &uart0;
+ serial1 = &uart6;
+ serial2 = &uart7;
spi0 = &spi1;
spi1 = &spi2;
};
aliases {
ethernet0 = &gmac;
- serial0 = &uart0;
- serial1 = &uart1;
- serial2 = &uart2;
- serial3 = &uart3;
- serial4 = &uart4;
- serial5 = &uart5;
- serial6 = &uart6;
- serial7 = &uart7;
};
chosen {
model = "Ippo Q8H Dual Core Tablet (v5)";
compatible = "ippo,q8h-v5", "allwinner,sun8i-a23";
+ aliases {
+ serial0 = &r_uart;
+ };
+
chosen {
bootargs = "earlyprintk console=ttyS0,115200";
};
/ {
interrupt-parent = <&gic>;
- aliases {
- serial0 = &uart0;
- serial1 = &uart1;
- serial2 = &uart2;
- serial3 = &uart3;
- serial4 = &uart4;
- serial5 = &r_uart;
- };
-
cpus {
#address-cells = <1>;
#size-cells = <0>;
model = "Merrii A80 Optimus Board";
compatible = "merrii,a80-optimus", "allwinner,sun9i-a80";
+ aliases {
+ serial0 = &uart0;
+ serial1 = &uart4;
+ };
+
chosen {
bootargs = "earlyprintk console=ttyS0,115200";
};
/ {
interrupt-parent = <&gic>;
- aliases {
- serial0 = &uart0;
- serial1 = &uart1;
- serial2 = &uart2;
- serial3 = &uart3;
- serial4 = &uart4;
- serial5 = &uart5;
- serial6 = &r_uart;
- };
-
cpus {
#address-cells = <1>;
#size-cells = <0>;
vcpu->arch.hcr = HCR_GUEST_MASK;
}
+static inline unsigned long vcpu_get_hcr(struct kvm_vcpu *vcpu)
+{
+ return vcpu->arch.hcr;
+}
+
+static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long hcr)
+{
+ vcpu->arch.hcr = hcr;
+}
+
static inline bool vcpu_mode_is_32bit(struct kvm_vcpu *vcpu)
{
return 1;
* Anything that is not used directly from assembly code goes
* here.
*/
- /* dcache set/way operation pending */
- int last_pcpu;
- cpumask_t require_dcache_flush;
/* Don't run the guest on this vcpu */
bool pause;
#ifndef __ASSEMBLY__
+#include <linux/highmem.h>
#include <asm/cacheflush.h>
#include <asm/pgalloc.h>
return (vcpu->arch.cp15[c1_SCTLR] & 0b101) == 0b101;
}
-static inline void coherent_cache_guest_page(struct kvm_vcpu *vcpu, hva_t hva,
- unsigned long size,
- bool ipa_uncached)
+static inline void __coherent_cache_guest_page(struct kvm_vcpu *vcpu, pfn_t pfn,
+ unsigned long size,
+ bool ipa_uncached)
{
- if (!vcpu_has_cache_enabled(vcpu) || ipa_uncached)
- kvm_flush_dcache_to_poc((void *)hva, size);
-
/*
* If we are going to insert an instruction page and the icache is
* either VIPT or PIPT, there is a potential problem where the host
*
* VIVT caches are tagged using both the ASID and the VMID and doesn't
* need any kind of flushing (DDI 0406C.b - Page B3-1392).
+ *
+ * We need to do this through a kernel mapping (using the
+ * user-space mapping has proved to be the wrong
+ * solution). For that, we need to kmap one page at a time,
+ * and iterate over the range.
*/
- if (icache_is_pipt()) {
- __cpuc_coherent_user_range(hva, hva + size);
- } else if (!icache_is_vivt_asid_tagged()) {
+
+ bool need_flush = !vcpu_has_cache_enabled(vcpu) || ipa_uncached;
+
+ VM_BUG_ON(size & PAGE_MASK);
+
+ if (!need_flush && !icache_is_pipt())
+ goto vipt_cache;
+
+ while (size) {
+ void *va = kmap_atomic_pfn(pfn);
+
+ if (need_flush)
+ kvm_flush_dcache_to_poc(va, PAGE_SIZE);
+
+ if (icache_is_pipt())
+ __cpuc_coherent_user_range((unsigned long)va,
+ (unsigned long)va + PAGE_SIZE);
+
+ size -= PAGE_SIZE;
+ pfn++;
+
+ kunmap_atomic(va);
+ }
+
+vipt_cache:
+ if (!icache_is_pipt() && !icache_is_vivt_asid_tagged()) {
/* any kind of VIPT cache */
__flush_icache_all();
}
}
+static inline void __kvm_flush_dcache_pte(pte_t pte)
+{
+ void *va = kmap_atomic(pte_page(pte));
+
+ kvm_flush_dcache_to_poc(va, PAGE_SIZE);
+
+ kunmap_atomic(va);
+}
+
+static inline void __kvm_flush_dcache_pmd(pmd_t pmd)
+{
+ unsigned long size = PMD_SIZE;
+ pfn_t pfn = pmd_pfn(pmd);
+
+ while (size) {
+ void *va = kmap_atomic_pfn(pfn);
+
+ kvm_flush_dcache_to_poc(va, PAGE_SIZE);
+
+ pfn++;
+ size -= PAGE_SIZE;
+
+ kunmap_atomic(va);
+ }
+}
+
+static inline void __kvm_flush_dcache_pud(pud_t pud)
+{
+}
+
#define kvm_virt_to_phys(x) virt_to_idmap((unsigned long)(x))
-void stage2_flush_vm(struct kvm *kvm);
+void kvm_set_way_flush(struct kvm_vcpu *vcpu);
+void kvm_toggle_cache(struct kvm_vcpu *vcpu, bool was_enabled);
#endif /* !__ASSEMBLY__ */
__invalid_entry:
v7m_exception_entry
+#ifdef CONFIG_PRINTK
adr r0, strerr
mrs r1, ipsr
mov r2, lr
bl printk
+#endif
mov r0, sp
bl show_regs
1: b 1b
select HAVE_KVM_CPU_RELAX_INTERCEPT
select KVM_MMIO
select KVM_ARM_HOST
+ select SRCU
depends on ARM_VIRT_EXT && ARM_LPAE
---help---
Support hosting virtualized guest machines. You will also
vcpu->cpu = cpu;
vcpu->arch.host_cpu_context = this_cpu_ptr(kvm_host_cpu_state);
- /*
- * Check whether this vcpu requires the cache to be flushed on
- * this physical CPU. This is a consequence of doing dcache
- * operations by set/way on this vcpu. We do it here to be in
- * a non-preemptible section.
- */
- if (cpumask_test_and_clear_cpu(cpu, &vcpu->arch.require_dcache_flush))
- flush_cache_all(); /* We'd really want v7_flush_dcache_all() */
-
kvm_arm_set_running_vcpu(vcpu);
}
ret = kvm_call_hyp(__kvm_vcpu_run, vcpu);
vcpu->mode = OUTSIDE_GUEST_MODE;
- vcpu->arch.last_pcpu = smp_processor_id();
kvm_guest_exit();
trace_kvm_exit(*vcpu_pc(vcpu));
/*
return true;
}
-/* See note at ARM ARM B1.14.4 */
+/*
+ * See note at ARMv7 ARM B1.14.4 (TL;DR: S/W ops are not easily virtualized).
+ */
static bool access_dcsw(struct kvm_vcpu *vcpu,
const struct coproc_params *p,
const struct coproc_reg *r)
{
- unsigned long val;
- int cpu;
-
if (!p->is_write)
return read_from_write_only(vcpu, p);
- cpu = get_cpu();
-
- cpumask_setall(&vcpu->arch.require_dcache_flush);
- cpumask_clear_cpu(cpu, &vcpu->arch.require_dcache_flush);
-
- /* If we were already preempted, take the long way around */
- if (cpu != vcpu->arch.last_pcpu) {
- flush_cache_all();
- goto done;
- }
-
- val = *vcpu_reg(vcpu, p->Rt1);
-
- switch (p->CRm) {
- case 6: /* Upgrade DCISW to DCCISW, as per HCR.SWIO */
- case 14: /* DCCISW */
- asm volatile("mcr p15, 0, %0, c7, c14, 2" : : "r" (val));
- break;
-
- case 10: /* DCCSW */
- asm volatile("mcr p15, 0, %0, c7, c10, 2" : : "r" (val));
- break;
- }
-
-done:
- put_cpu();
-
+ kvm_set_way_flush(vcpu);
return true;
}
/*
* Generic accessor for VM registers. Only called as long as HCR_TVM
- * is set.
+ * is set. If the guest enables the MMU, we stop trapping the VM
+ * sys_regs and leave it in complete control of the caches.
+ *
+ * Used by the cpu-specific code.
*/
-static bool access_vm_reg(struct kvm_vcpu *vcpu,
- const struct coproc_params *p,
- const struct coproc_reg *r)
+bool access_vm_reg(struct kvm_vcpu *vcpu,
+ const struct coproc_params *p,
+ const struct coproc_reg *r)
{
+ bool was_enabled = vcpu_has_cache_enabled(vcpu);
+
BUG_ON(!p->is_write);
vcpu->arch.cp15[r->reg] = *vcpu_reg(vcpu, p->Rt1);
if (p->is_64bit)
vcpu->arch.cp15[r->reg + 1] = *vcpu_reg(vcpu, p->Rt2);
- return true;
-}
-
-/*
- * SCTLR accessor. Only called as long as HCR_TVM is set. If the
- * guest enables the MMU, we stop trapping the VM sys_regs and leave
- * it in complete control of the caches.
- *
- * Used by the cpu-specific code.
- */
-bool access_sctlr(struct kvm_vcpu *vcpu,
- const struct coproc_params *p,
- const struct coproc_reg *r)
-{
- access_vm_reg(vcpu, p, r);
-
- if (vcpu_has_cache_enabled(vcpu)) { /* MMU+Caches enabled? */
- vcpu->arch.hcr &= ~HCR_TVM;
- stage2_flush_vm(vcpu->kvm);
- }
-
+ kvm_toggle_cache(vcpu, was_enabled);
return true;
}
#define is64 .is_64 = true
#define is32 .is_64 = false
-bool access_sctlr(struct kvm_vcpu *vcpu,
- const struct coproc_params *p,
- const struct coproc_reg *r);
+bool access_vm_reg(struct kvm_vcpu *vcpu,
+ const struct coproc_params *p,
+ const struct coproc_reg *r);
#endif /* __ARM_KVM_COPROC_LOCAL_H__ */
static const struct coproc_reg a15_regs[] = {
/* SCTLR: swapped by interrupt.S. */
{ CRn( 1), CRm( 0), Op1( 0), Op2( 0), is32,
- access_sctlr, reset_val, c1_SCTLR, 0x00C50078 },
+ access_vm_reg, reset_val, c1_SCTLR, 0x00C50078 },
};
static struct kvm_coproc_target_table a15_target_table = {
static const struct coproc_reg a7_regs[] = {
/* SCTLR: swapped by interrupt.S. */
{ CRn( 1), CRm( 0), Op1( 0), Op2( 0), is32,
- access_sctlr, reset_val, c1_SCTLR, 0x00C50878 },
+ access_vm_reg, reset_val, c1_SCTLR, 0x00C50878 },
};
static struct kvm_coproc_target_table a7_target_table = {
kvm_call_hyp(__kvm_tlb_flush_vmid_ipa, kvm, ipa);
}
+/*
+ * D-Cache management functions. They take the page table entries by
+ * value, as they are flushing the cache using the kernel mapping (or
+ * kmap on 32bit).
+ */
+static void kvm_flush_dcache_pte(pte_t pte)
+{
+ __kvm_flush_dcache_pte(pte);
+}
+
+static void kvm_flush_dcache_pmd(pmd_t pmd)
+{
+ __kvm_flush_dcache_pmd(pmd);
+}
+
+static void kvm_flush_dcache_pud(pud_t pud)
+{
+ __kvm_flush_dcache_pud(pud);
+}
+
static int mmu_topup_memory_cache(struct kvm_mmu_memory_cache *cache,
int min, int max)
{
put_page(virt_to_page(pmd));
}
+/*
+ * Unmapping vs dcache management:
+ *
+ * If a guest maps certain memory pages as uncached, all writes will
+ * bypass the data cache and go directly to RAM. However, the CPUs
+ * can still speculate reads (not writes) and fill cache lines with
+ * data.
+ *
+ * Those cache lines will be *clean* cache lines though, so a
+ * clean+invalidate operation is equivalent to an invalidate
+ * operation, because no cache lines are marked dirty.
+ *
+ * Those clean cache lines could be filled prior to an uncached write
+ * by the guest, and the cache coherent IO subsystem would therefore
+ * end up writing old data to disk.
+ *
+ * This is why right after unmapping a page/section and invalidating
+ * the corresponding TLBs, we call kvm_flush_dcache_p*() to make sure
+ * the IO subsystem will never hit in the cache.
+ */
static void unmap_ptes(struct kvm *kvm, pmd_t *pmd,
phys_addr_t addr, phys_addr_t end)
{
start_pte = pte = pte_offset_kernel(pmd, addr);
do {
if (!pte_none(*pte)) {
+ pte_t old_pte = *pte;
+
kvm_set_pte(pte, __pte(0));
- put_page(virt_to_page(pte));
kvm_tlb_flush_vmid_ipa(kvm, addr);
+
+ /* No need to invalidate the cache for device mappings */
+ if ((pte_val(old_pte) & PAGE_S2_DEVICE) != PAGE_S2_DEVICE)
+ kvm_flush_dcache_pte(old_pte);
+
+ put_page(virt_to_page(pte));
}
} while (pte++, addr += PAGE_SIZE, addr != end);
next = kvm_pmd_addr_end(addr, end);
if (!pmd_none(*pmd)) {
if (kvm_pmd_huge(*pmd)) {
+ pmd_t old_pmd = *pmd;
+
pmd_clear(pmd);
kvm_tlb_flush_vmid_ipa(kvm, addr);
+
+ kvm_flush_dcache_pmd(old_pmd);
+
put_page(virt_to_page(pmd));
} else {
unmap_ptes(kvm, pmd, addr, next);
next = kvm_pud_addr_end(addr, end);
if (!pud_none(*pud)) {
if (pud_huge(*pud)) {
+ pud_t old_pud = *pud;
+
pud_clear(pud);
kvm_tlb_flush_vmid_ipa(kvm, addr);
+
+ kvm_flush_dcache_pud(old_pud);
+
put_page(virt_to_page(pud));
} else {
unmap_pmds(kvm, pud, addr, next);
pte = pte_offset_kernel(pmd, addr);
do {
- if (!pte_none(*pte)) {
- hva_t hva = gfn_to_hva(kvm, addr >> PAGE_SHIFT);
- kvm_flush_dcache_to_poc((void*)hva, PAGE_SIZE);
- }
+ if (!pte_none(*pte) &&
+ (pte_val(*pte) & PAGE_S2_DEVICE) != PAGE_S2_DEVICE)
+ kvm_flush_dcache_pte(*pte);
} while (pte++, addr += PAGE_SIZE, addr != end);
}
do {
next = kvm_pmd_addr_end(addr, end);
if (!pmd_none(*pmd)) {
- if (kvm_pmd_huge(*pmd)) {
- hva_t hva = gfn_to_hva(kvm, addr >> PAGE_SHIFT);
- kvm_flush_dcache_to_poc((void*)hva, PMD_SIZE);
- } else {
+ if (kvm_pmd_huge(*pmd))
+ kvm_flush_dcache_pmd(*pmd);
+ else
stage2_flush_ptes(kvm, pmd, addr, next);
- }
}
} while (pmd++, addr = next, addr != end);
}
do {
next = kvm_pud_addr_end(addr, end);
if (!pud_none(*pud)) {
- if (pud_huge(*pud)) {
- hva_t hva = gfn_to_hva(kvm, addr >> PAGE_SHIFT);
- kvm_flush_dcache_to_poc((void*)hva, PUD_SIZE);
- } else {
+ if (pud_huge(*pud))
+ kvm_flush_dcache_pud(*pud);
+ else
stage2_flush_pmds(kvm, pud, addr, next);
- }
}
} while (pud++, addr = next, addr != end);
}
* Go through the stage 2 page tables and invalidate any cache lines
* backing memory already mapped to the VM.
*/
-void stage2_flush_vm(struct kvm *kvm)
+static void stage2_flush_vm(struct kvm *kvm)
{
struct kvm_memslots *slots;
struct kvm_memory_slot *memslot;
return !pfn_valid(pfn);
}
+static void coherent_cache_guest_page(struct kvm_vcpu *vcpu, pfn_t pfn,
+ unsigned long size, bool uncached)
+{
+ __coherent_cache_guest_page(vcpu, pfn, size, uncached);
+}
+
static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
struct kvm_memory_slot *memslot, unsigned long hva,
unsigned long fault_status)
kvm_set_s2pmd_writable(&new_pmd);
kvm_set_pfn_dirty(pfn);
}
- coherent_cache_guest_page(vcpu, hva & PMD_MASK, PMD_SIZE,
- fault_ipa_uncached);
+ coherent_cache_guest_page(vcpu, pfn, PMD_SIZE, fault_ipa_uncached);
ret = stage2_set_pmd_huge(kvm, memcache, fault_ipa, &new_pmd);
} else {
pte_t new_pte = pfn_pte(pfn, mem_type);
kvm_set_s2pte_writable(&new_pte);
kvm_set_pfn_dirty(pfn);
}
- coherent_cache_guest_page(vcpu, hva, PAGE_SIZE,
- fault_ipa_uncached);
+ coherent_cache_guest_page(vcpu, pfn, PAGE_SIZE, fault_ipa_uncached);
ret = stage2_set_pte(kvm, memcache, fault_ipa, &new_pte,
pgprot_val(mem_type) == pgprot_val(PAGE_S2_DEVICE));
}
unmap_stage2_range(kvm, gpa, size);
spin_unlock(&kvm->mmu_lock);
}
+
+/*
+ * See note at ARMv7 ARM B1.14.4 (TL;DR: S/W ops are not easily virtualized).
+ *
+ * Main problems:
+ * - S/W ops are local to a CPU (not broadcast)
+ * - We have line migration behind our back (speculation)
+ * - System caches don't support S/W at all (damn!)
+ *
+ * In the face of the above, the best we can do is to try and convert
+ * S/W ops to VA ops. Because the guest is not allowed to infer the
+ * S/W to PA mapping, it can only use S/W to nuke the whole cache,
+ * which is a rather good thing for us.
+ *
+ * Also, it is only used when turning caches on/off ("The expected
+ * usage of the cache maintenance instructions that operate by set/way
+ * is associated with the cache maintenance instructions associated
+ * with the powerdown and powerup of caches, if this is required by
+ * the implementation.").
+ *
+ * We use the following policy:
+ *
+ * - If we trap a S/W operation, we enable VM trapping to detect
+ * caches being turned on/off, and do a full clean.
+ *
+ * - We flush the caches on both caches being turned on and off.
+ *
+ * - Once the caches are enabled, we stop trapping VM ops.
+ */
+void kvm_set_way_flush(struct kvm_vcpu *vcpu)
+{
+ unsigned long hcr = vcpu_get_hcr(vcpu);
+
+ /*
+ * If this is the first time we do a S/W operation
+ * (i.e. HCR_TVM not set) flush the whole memory, and set the
+ * VM trapping.
+ *
+ * Otherwise, rely on the VM trapping to wait for the MMU +
+ * Caches to be turned off. At that point, we'll be able to
+ * clean the caches again.
+ */
+ if (!(hcr & HCR_TVM)) {
+ trace_kvm_set_way_flush(*vcpu_pc(vcpu),
+ vcpu_has_cache_enabled(vcpu));
+ stage2_flush_vm(vcpu->kvm);
+ vcpu_set_hcr(vcpu, hcr | HCR_TVM);
+ }
+}
+
+void kvm_toggle_cache(struct kvm_vcpu *vcpu, bool was_enabled)
+{
+ bool now_enabled = vcpu_has_cache_enabled(vcpu);
+
+ /*
+ * If switching the MMU+caches on, need to invalidate the caches.
+ * If switching it off, need to clean the caches.
+ * Clean + invalidate does the trick always.
+ */
+ if (now_enabled != was_enabled)
+ stage2_flush_vm(vcpu->kvm);
+
+ /* Caches are now on, stop trapping VM ops (until a S/W op) */
+ if (now_enabled)
+ vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) & ~HCR_TVM);
+
+ trace_kvm_toggle_cache(*vcpu_pc(vcpu), was_enabled, now_enabled);
+}
__entry->vcpu_pc, __entry->r0, __entry->imm)
);
+TRACE_EVENT(kvm_set_way_flush,
+ TP_PROTO(unsigned long vcpu_pc, bool cache),
+ TP_ARGS(vcpu_pc, cache),
+
+ TP_STRUCT__entry(
+ __field( unsigned long, vcpu_pc )
+ __field( bool, cache )
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu_pc = vcpu_pc;
+ __entry->cache = cache;
+ ),
+
+ TP_printk("S/W flush at 0x%016lx (cache %s)",
+ __entry->vcpu_pc, __entry->cache ? "on" : "off")
+);
+
+TRACE_EVENT(kvm_toggle_cache,
+ TP_PROTO(unsigned long vcpu_pc, bool was, bool now),
+ TP_ARGS(vcpu_pc, was, now),
+
+ TP_STRUCT__entry(
+ __field( unsigned long, vcpu_pc )
+ __field( bool, was )
+ __field( bool, now )
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu_pc = vcpu_pc;
+ __entry->was = was;
+ __entry->now = now;
+ ),
+
+ TP_printk("VM op at 0x%016lx (cache was %s, now %s)",
+ __entry->vcpu_pc, __entry->was ? "on" : "off",
+ __entry->now ? "on" : "off")
+);
+
#endif /* _TRACE_KVM_H */
#undef TRACE_INCLUDE_PATH
coherency_cpu_base = of_iomap(np, 0);
arch_ioremap_caller = armada_pcie_wa_ioremap_caller;
+ /*
+ * We should switch the PL310 to I/O coherency mode only if
+ * I/O coherency is actually enabled.
+ */
+ if (!coherency_available())
+ return;
+
/*
* Add the PL310 property "arm,io-coherent". This makes sure the
* outer sync operation is not used, which allows to
#include <linux/gpio_keys.h>
#include <linux/input.h>
#include <linux/interrupt.h>
+#include <linux/irqchip.h>
+#include <linux/irqchip/arm-gic.h>
#include <linux/kernel.h>
#include <linux/mfd/tmio.h>
#include <linux/mmc/host.h>
sizeof(ape6evm_leds_pdata));
}
+static void __init ape6evm_legacy_init_time(void)
+{
+ /* Do not invoke DT-based timers via clocksource_of_init() */
+}
+
+static void __init ape6evm_legacy_init_irq(void)
+{
+ void __iomem *gic_dist_base = ioremap_nocache(0xf1001000, 0x1000);
+ void __iomem *gic_cpu_base = ioremap_nocache(0xf1002000, 0x1000);
+
+ gic_init(0, 29, gic_dist_base, gic_cpu_base);
+
+ /* Do not invoke DT-based interrupt code via irqchip_init() */
+}
+
+
static const char *ape6evm_boards_compat_dt[] __initdata = {
"renesas,ape6evm",
NULL,
DT_MACHINE_START(APE6EVM_DT, "ape6evm")
.init_early = shmobile_init_delay,
+ .init_irq = ape6evm_legacy_init_irq,
.init_machine = ape6evm_add_standard_devices,
.init_late = shmobile_init_late,
.dt_compat = ape6evm_boards_compat_dt,
+ .init_time = ape6evm_legacy_init_time,
MACHINE_END
#include <linux/input.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
+#include <linux/irqchip.h>
+#include <linux/irqchip/arm-gic.h>
#include <linux/kernel.h>
#include <linux/leds.h>
#include <linux/mfd/tmio.h>
lager_ksz8041_fixup);
}
+static void __init lager_legacy_init_irq(void)
+{
+ void __iomem *gic_dist_base = ioremap_nocache(0xf1001000, 0x1000);
+ void __iomem *gic_cpu_base = ioremap_nocache(0xf1002000, 0x1000);
+
+ gic_init(0, 29, gic_dist_base, gic_cpu_base);
+
+ /* Do not invoke DT-based interrupt code via irqchip_init() */
+}
+
static const char * const lager_boards_compat_dt[] __initconst = {
"renesas,lager",
NULL,
DT_MACHINE_START(LAGER_DT, "lager")
.smp = smp_ops(r8a7790_smp_ops),
.init_early = shmobile_init_delay,
+ .init_irq = lager_legacy_init_irq,
.init_time = rcar_gen2_timer_init,
.init_machine = lager_init,
.init_late = shmobile_init_late,
#ifdef CONFIG_COMMON_CLK
rcar_gen2_clocks_init(mode);
#endif
+#ifdef CONFIG_ARCH_SHMOBILE_MULTI
clocksource_of_init();
+#endif
}
struct memory_reserve_config {
if (!max_freq)
return;
+#ifdef CONFIG_ARCH_SHMOBILE_LEGACY
+ /* Non-multiplatform r8a73a4 SoC cannot use arch timer due
+ * to GIC being initialized from C and arch timer via DT */
+ if (of_machine_is_compatible("renesas,r8a73a4"))
+ has_arch_timer = false;
+
+ /* Non-multiplatform r8a7790 SoC cannot use arch timer due
+ * to GIC being initialized from C and arch timer via DT */
+ if (of_machine_is_compatible("renesas,r8a7790"))
+ has_arch_timer = false;
+#endif
+
if (!has_arch_timer || !IS_ENABLED(CONFIG_ARM_ARCH_TIMER)) {
if (is_a7_a8_a9)
shmobile_setup_delay_hz(max_freq, 1, 3);
config ARM_KERNMEM_PERMS
bool "Restrict kernel memory permissions"
+ depends on MMU
help
If this is set, kernel memory other than kernel text (and rodata)
will be made non-executable. The tradeoff is that each region is
/* Update the list of reserved ASIDs and the ASID bitmap. */
bitmap_clear(asid_map, 0, NUM_USER_ASIDS);
for_each_possible_cpu(i) {
- if (i == cpu) {
- asid = 0;
- } else {
- asid = atomic64_xchg(&per_cpu(active_asids, i), 0);
- /*
- * If this CPU has already been through a
- * rollover, but hasn't run another task in
- * the meantime, we must preserve its reserved
- * ASID, as this is the only trace we have of
- * the process it is still running.
- */
- if (asid == 0)
- asid = per_cpu(reserved_asids, i);
- __set_bit(asid & ~ASID_MASK, asid_map);
- }
+ asid = atomic64_xchg(&per_cpu(active_asids, i), 0);
+ /*
+ * If this CPU has already been through a
+ * rollover, but hasn't run another task in
+ * the meantime, we must preserve its reserved
+ * ASID, as this is the only trace we have of
+ * the process it is still running.
+ */
+ if (asid == 0)
+ asid = per_cpu(reserved_asids, i);
+ __set_bit(asid & ~ASID_MASK, asid_map);
per_cpu(reserved_asids, i) = asid;
}
}
EXPORT_SYMBOL_GPL(arm_iommu_release_mapping);
+static int __arm_iommu_attach_device(struct device *dev,
+ struct dma_iommu_mapping *mapping)
+{
+ int err;
+
+ err = iommu_attach_device(mapping->domain, dev);
+ if (err)
+ return err;
+
+ kref_get(&mapping->kref);
+ dev->archdata.mapping = mapping;
+
+ pr_debug("Attached IOMMU controller to %s device.\n", dev_name(dev));
+ return 0;
+}
+
/**
* arm_iommu_attach_device
* @dev: valid struct device pointer
* @mapping: io address space mapping structure (returned from
* arm_iommu_create_mapping)
*
- * Attaches specified io address space mapping to the provided device,
+ * Attaches specified io address space mapping to the provided device.
+ * This replaces the dma operations (dma_map_ops pointer) with the
+ * IOMMU aware version.
+ *
* More than one client might be attached to the same io address space
* mapping.
*/
{
int err;
- err = iommu_attach_device(mapping->domain, dev);
+ err = __arm_iommu_attach_device(dev, mapping);
if (err)
return err;
- kref_get(&mapping->kref);
- dev->archdata.mapping = mapping;
-
- pr_debug("Attached IOMMU controller to %s device.\n", dev_name(dev));
+ set_dma_ops(dev, &iommu_ops);
return 0;
}
EXPORT_SYMBOL_GPL(arm_iommu_attach_device);
-/**
- * arm_iommu_detach_device
- * @dev: valid struct device pointer
- *
- * Detaches the provided device from a previously attached map.
- */
-void arm_iommu_detach_device(struct device *dev)
+static void __arm_iommu_detach_device(struct device *dev)
{
struct dma_iommu_mapping *mapping;
pr_debug("Detached IOMMU controller from %s device.\n", dev_name(dev));
}
+
+/**
+ * arm_iommu_detach_device
+ * @dev: valid struct device pointer
+ *
+ * Detaches the provided device from a previously attached map.
+ * This voids the dma operations (dma_map_ops pointer)
+ */
+void arm_iommu_detach_device(struct device *dev)
+{
+ __arm_iommu_detach_device(dev);
+ set_dma_ops(dev, NULL);
+}
EXPORT_SYMBOL_GPL(arm_iommu_detach_device);
static struct dma_map_ops *arm_get_iommu_dma_map_ops(bool coherent)
return false;
}
- if (arm_iommu_attach_device(dev, mapping)) {
+ if (__arm_iommu_attach_device(dev, mapping)) {
pr_warn("Failed to attached device %s to IOMMU_mapping\n",
dev_name(dev));
arm_iommu_release_mapping(mapping);
{
struct dma_iommu_mapping *mapping = dev->archdata.mapping;
- arm_iommu_detach_device(dev);
+ if (!mapping)
+ return;
+
+ __arm_iommu_detach_device(dev);
arm_iommu_release_mapping(mapping);
}
vcpu->arch.hcr_el2 &= ~HCR_RW;
}
+static inline unsigned long vcpu_get_hcr(struct kvm_vcpu *vcpu)
+{
+ return vcpu->arch.hcr_el2;
+}
+
+static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long hcr)
+{
+ vcpu->arch.hcr_el2 = hcr;
+}
+
static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
{
return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
* Anything that is not used directly from assembly code goes
* here.
*/
- /* dcache set/way operation pending */
- int last_pcpu;
- cpumask_t require_dcache_flush;
/* Don't run the guest */
bool pause;
return (vcpu_sys_reg(vcpu, SCTLR_EL1) & 0b101) == 0b101;
}
-static inline void coherent_cache_guest_page(struct kvm_vcpu *vcpu, hva_t hva,
- unsigned long size,
- bool ipa_uncached)
+static inline void __coherent_cache_guest_page(struct kvm_vcpu *vcpu, pfn_t pfn,
+ unsigned long size,
+ bool ipa_uncached)
{
+ void *va = page_address(pfn_to_page(pfn));
+
if (!vcpu_has_cache_enabled(vcpu) || ipa_uncached)
- kvm_flush_dcache_to_poc((void *)hva, size);
+ kvm_flush_dcache_to_poc(va, size);
if (!icache_is_aliasing()) { /* PIPT */
- flush_icache_range(hva, hva + size);
+ flush_icache_range((unsigned long)va,
+ (unsigned long)va + size);
} else if (!icache_is_aivivt()) { /* non ASID-tagged VIVT */
/* any kind of VIPT cache */
__flush_icache_all();
}
}
+static inline void __kvm_flush_dcache_pte(pte_t pte)
+{
+ struct page *page = pte_page(pte);
+ kvm_flush_dcache_to_poc(page_address(page), PAGE_SIZE);
+}
+
+static inline void __kvm_flush_dcache_pmd(pmd_t pmd)
+{
+ struct page *page = pmd_page(pmd);
+ kvm_flush_dcache_to_poc(page_address(page), PMD_SIZE);
+}
+
+static inline void __kvm_flush_dcache_pud(pud_t pud)
+{
+ struct page *page = pud_page(pud);
+ kvm_flush_dcache_to_poc(page_address(page), PUD_SIZE);
+}
+
#define kvm_virt_to_phys(x) __virt_to_phys((unsigned long)(x))
-void stage2_flush_vm(struct kvm *kvm);
+void kvm_set_way_flush(struct kvm_vcpu *vcpu);
+void kvm_toggle_cache(struct kvm_vcpu *vcpu, bool was_enabled);
#endif /* __ASSEMBLY__ */
#endif /* __ARM64_KVM_MMU_H__ */
select KVM_ARM_HOST
select KVM_ARM_VGIC
select KVM_ARM_TIMER
+ select SRCU
---help---
Support hosting virtualized guest machines.
return ccsidr;
}
-static void do_dc_cisw(u32 val)
-{
- asm volatile("dc cisw, %x0" : : "r" (val));
- dsb(ish);
-}
-
-static void do_dc_csw(u32 val)
-{
- asm volatile("dc csw, %x0" : : "r" (val));
- dsb(ish);
-}
-
-/* See note at ARM ARM B1.14.4 */
+/*
+ * See note at ARMv7 ARM B1.14.4 (TL;DR: S/W ops are not easily virtualized).
+ */
static bool access_dcsw(struct kvm_vcpu *vcpu,
const struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
- unsigned long val;
- int cpu;
-
if (!p->is_write)
return read_from_write_only(vcpu, p);
- cpu = get_cpu();
-
- cpumask_setall(&vcpu->arch.require_dcache_flush);
- cpumask_clear_cpu(cpu, &vcpu->arch.require_dcache_flush);
-
- /* If we were already preempted, take the long way around */
- if (cpu != vcpu->arch.last_pcpu) {
- flush_cache_all();
- goto done;
- }
-
- val = *vcpu_reg(vcpu, p->Rt);
-
- switch (p->CRm) {
- case 6: /* Upgrade DCISW to DCCISW, as per HCR.SWIO */
- case 14: /* DCCISW */
- do_dc_cisw(val);
- break;
-
- case 10: /* DCCSW */
- do_dc_csw(val);
- break;
- }
-
-done:
- put_cpu();
-
+ kvm_set_way_flush(vcpu);
return true;
}
/*
* Generic accessor for VM registers. Only called as long as HCR_TVM
- * is set.
+ * is set. If the guest enables the MMU, we stop trapping the VM
+ * sys_regs and leave it in complete control of the caches.
*/
static bool access_vm_reg(struct kvm_vcpu *vcpu,
const struct sys_reg_params *p,
const struct sys_reg_desc *r)
{
unsigned long val;
+ bool was_enabled = vcpu_has_cache_enabled(vcpu);
BUG_ON(!p->is_write);
vcpu_cp15_64_low(vcpu, r->reg) = val & 0xffffffffUL;
}
- return true;
-}
-
-/*
- * SCTLR_EL1 accessor. Only called as long as HCR_TVM is set. If the
- * guest enables the MMU, we stop trapping the VM sys_regs and leave
- * it in complete control of the caches.
- */
-static bool access_sctlr(struct kvm_vcpu *vcpu,
- const struct sys_reg_params *p,
- const struct sys_reg_desc *r)
-{
- access_vm_reg(vcpu, p, r);
-
- if (vcpu_has_cache_enabled(vcpu)) { /* MMU+Caches enabled? */
- vcpu->arch.hcr_el2 &= ~HCR_TVM;
- stage2_flush_vm(vcpu->kvm);
- }
-
+ kvm_toggle_cache(vcpu, was_enabled);
return true;
}
NULL, reset_mpidr, MPIDR_EL1 },
/* SCTLR_EL1 */
{ Op0(0b11), Op1(0b000), CRn(0b0001), CRm(0b0000), Op2(0b000),
- access_sctlr, reset_val, SCTLR_EL1, 0x00C50078 },
+ access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 },
/* CPACR_EL1 */
{ Op0(0b11), Op1(0b000), CRn(0b0001), CRm(0b0000), Op2(0b010),
NULL, reset_val, CPACR_EL1, 0 },
* register).
*/
static const struct sys_reg_desc cp15_regs[] = {
- { Op1( 0), CRn( 1), CRm( 0), Op2( 0), access_sctlr, NULL, c1_SCTLR },
+ { Op1( 0), CRn( 1), CRm( 0), Op2( 0), access_vm_reg, NULL, c1_SCTLR },
{ Op1( 0), CRn( 2), CRm( 0), Op2( 0), access_vm_reg, NULL, c2_TTBR0 },
{ Op1( 0), CRn( 2), CRm( 0), Op2( 1), access_vm_reg, NULL, c2_TTBR1 },
{ Op1( 0), CRn( 2), CRm( 0), Op2( 2), access_vm_reg, NULL, c2_TTBCR },
if (unlikely(fault & VM_FAULT_ERROR)) {
if (fault & VM_FAULT_OOM)
goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
else if (fault & VM_FAULT_SIGBUS)
goto do_sigbus;
BUG();
if (unlikely(fault & VM_FAULT_ERROR)) {
if (fault & VM_FAULT_OOM)
goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
else if (fault & VM_FAULT_SIGBUS)
goto do_sigbus;
BUG();
if (unlikely(fault & VM_FAULT_ERROR)) {
if (fault & VM_FAULT_OOM)
goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
else if (fault & VM_FAULT_SIGBUS)
goto do_sigbus;
BUG();
*/
if (fault & VM_FAULT_OOM) {
goto out_of_memory;
+ } else if (fault & VM_FAULT_SIGSEGV) {
+ goto bad_area;
} else if (fault & VM_FAULT_SIGBUS) {
signal = SIGBUS;
goto bad_area;
if (unlikely(fault & VM_FAULT_ERROR)) {
if (fault & VM_FAULT_OOM)
goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
else if (fault & VM_FAULT_SIGBUS)
goto do_sigbus;
BUG();
if (unlikely(fault & VM_FAULT_ERROR)) {
if (fault & VM_FAULT_OOM)
goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto map_err;
else if (fault & VM_FAULT_SIGBUS)
goto bus_err;
BUG();
if (unlikely(fault & VM_FAULT_ERROR)) {
if (fault & VM_FAULT_OOM)
goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
else if (fault & VM_FAULT_SIGBUS)
goto do_sigbus;
BUG();
if (unlikely(fault & VM_FAULT_ERROR)) {
if (fault & VM_FAULT_OOM)
goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
else if (fault & VM_FAULT_SIGBUS)
goto do_sigbus;
BUG();
bool
config MIPS32_COMPAT
- bool "Kernel support for Linux/MIPS 32-bit binary compatibility"
- depends on 64BIT
- help
- Select this option if you want Linux/MIPS 32-bit binary
- compatibility. Since all software available for Linux/MIPS is
- currently 32-bit you should say Y here.
+ bool
config COMPAT
bool
- depends on MIPS32_COMPAT
- select ARCH_WANT_OLD_COMPAT_IPC
- default y
config SYSVIPC_COMPAT
bool
- depends on COMPAT && SYSVIPC
- default y
config MIPS32_O32
bool "Kernel support for o32 binaries"
- depends on MIPS32_COMPAT
+ depends on 64BIT
+ select ARCH_WANT_OLD_COMPAT_IPC
+ select COMPAT
+ select MIPS32_COMPAT
+ select SYSVIPC_COMPAT if SYSVIPC
help
Select this option if you want to run o32 binaries. These are pure
32-bit binaries as used by the 32-bit Linux/MIPS port. Most of
config MIPS32_N32
bool "Kernel support for n32 binaries"
- depends on MIPS32_COMPAT
+ depends on 64BIT
+ select COMPAT
+ select MIPS32_COMPAT
+ select SYSVIPC_COMPAT if SYSVIPC
help
Select this option if you want to run n32 binaries. These are
64-bit binaries using 32-bit quantities for addressing and certain
/*
* Some extra ELF definitions
*/
-#define PT_MIPS_REGINFO 0x70000000 /* Register usage information */
+#define PT_MIPS_REGINFO 0x70000000 /* Register usage information */
+#define PT_MIPS_ABIFLAGS 0x70000003 /* Records ABI related flags */
/* -------------------------------------------------------------------- */
for (i = 0; i < ex.e_phnum; i++) {
/* Section types we can ignore... */
- if (ph[i].p_type == PT_NULL || ph[i].p_type == PT_NOTE ||
- ph[i].p_type == PT_PHDR
- || ph[i].p_type == PT_MIPS_REGINFO)
+ switch (ph[i].p_type) {
+ case PT_NULL:
+ case PT_NOTE:
+ case PT_PHDR:
+ case PT_MIPS_REGINFO:
+ case PT_MIPS_ABIFLAGS:
continue;
- /* Section types we can't handle... */
- else if (ph[i].p_type != PT_LOAD) {
- fprintf(stderr,
- "Program header %d type %d can't be converted.\n",
- ex.e_phnum, ph[i].p_type);
- exit(1);
- }
- /* Writable (data) segment? */
- if (ph[i].p_flags & PF_W) {
- struct sect ndata, nbss;
- ndata.vaddr = ph[i].p_vaddr;
- ndata.len = ph[i].p_filesz;
- nbss.vaddr = ph[i].p_vaddr + ph[i].p_filesz;
- nbss.len = ph[i].p_memsz - ph[i].p_filesz;
+ case PT_LOAD:
+ /* Writable (data) segment? */
+ if (ph[i].p_flags & PF_W) {
+ struct sect ndata, nbss;
+
+ ndata.vaddr = ph[i].p_vaddr;
+ ndata.len = ph[i].p_filesz;
+ nbss.vaddr = ph[i].p_vaddr + ph[i].p_filesz;
+ nbss.len = ph[i].p_memsz - ph[i].p_filesz;
- combine(&data, &ndata, 0);
- combine(&bss, &nbss, 1);
- } else {
- struct sect ntxt;
+ combine(&data, &ndata, 0);
+ combine(&bss, &nbss, 1);
+ } else {
+ struct sect ntxt;
- ntxt.vaddr = ph[i].p_vaddr;
- ntxt.len = ph[i].p_filesz;
+ ntxt.vaddr = ph[i].p_vaddr;
+ ntxt.len = ph[i].p_filesz;
- combine(&text, &ntxt, 0);
+ combine(&text, &ntxt, 0);
+ }
+ /* Remember the lowest segment start address. */
+ if (ph[i].p_vaddr < cur_vma)
+ cur_vma = ph[i].p_vaddr;
+ break;
+
+ default:
+ /* Section types we can't handle... */
+ fprintf(stderr,
+ "Program header %d type %d can't be converted.\n",
+ ex.e_phnum, ph[i].p_type);
+ exit(1);
}
- /* Remember the lowest segment start address. */
- if (ph[i].p_vaddr < cur_vma)
- cur_vma = ph[i].p_vaddr;
}
/* Sections must be in order to be converted... */
set_cpu_online(cpu, false);
cpu_clear(cpu, cpu_callin_map);
- local_irq_disable();
octeon_fixup_irqs();
- local_irq_enable();
flush_cache_all();
local_flush_tlb_all();
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
-CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_IP_NF_MANGLE=m
CONFIG_IP_NF_TARGET_CLUSTERIP=m
CONFIG_IP_NF_TARGET_ECN=m
CONFIG_BRIDGE_EBT_REDIRECT=m
CONFIG_BRIDGE_EBT_SNAT=m
CONFIG_BRIDGE_EBT_LOG=m
-CONFIG_BRIDGE_EBT_ULOG=m
CONFIG_BRIDGE_EBT_NFLOG=m
CONFIG_IP_SCTP=m
CONFIG_BRIDGE=m
CONFIG_NET_CLS_IND=y
CONFIG_CFG80211=m
CONFIG_MAC80211=m
-CONFIG_MAC80211_RC_PID=y
-CONFIG_MAC80211_RC_DEFAULT_PID=y
CONFIG_MAC80211_MESH=y
CONFIG_RFKILL=m
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_IDE=y
CONFIG_BLK_DEV_IDECD=y
CONFIG_IDE_GENERIC=y
-CONFIG_BLK_DEV_GENERIC=y
-CONFIG_BLK_DEV_PIIX=y
-CONFIG_BLK_DEV_IT8213=m
-CONFIG_BLK_DEV_TC86C001=m
CONFIG_RAID_ATTRS=m
-CONFIG_SCSI=m
-CONFIG_BLK_DEV_SD=m
+CONFIG_BLK_DEV_SD=y
CONFIG_CHR_DEV_ST=m
CONFIG_CHR_DEV_OSST=m
CONFIG_BLK_DEV_SR=m
CONFIG_BLK_DEV_SR_VENDOR=y
CONFIG_CHR_DEV_SG=m
-CONFIG_SCSI_MULTI_LUN=y
CONFIG_SCSI_CONSTANTS=y
CONFIG_SCSI_LOGGING=y
CONFIG_SCSI_SCAN_ASYNC=y
CONFIG_SCSI_AIC7XXX=m
CONFIG_AIC7XXX_RESET_DELAY_MS=15000
# CONFIG_AIC7XXX_DEBUG_ENABLE is not set
+CONFIG_ATA=y
+CONFIG_ATA_PIIX=y
CONFIG_MD=y
CONFIG_BLK_DEV_MD=m
CONFIG_MD_LINEAR=m
CONFIG_UIO_CIF=m
CONFIG_EXT2_FS=y
CONFIG_EXT3_FS=y
+CONFIG_EXT4_FS=y
CONFIG_REISERFS_FS=m
CONFIG_REISERFS_PROC_INFO=y
CONFIG_REISERFS_FS_XATTR=y
CONFIG_CRYPTO_TEA=m
CONFIG_CRYPTO_TWOFISH=m
# CONFIG_CRYPTO_ANSI_CPRNG is not set
-CONFIG_CRC16=m
return SIGFPE;
/* set FRE */
- write_c0_config5(read_c0_config5() | MIPS_CONF5_FRE);
+ set_c0_config5(MIPS_CONF5_FRE);
goto fr_common;
case FPU_64BIT:
#endif
/* fall through */
case FPU_32BIT:
- /* clear FRE */
- write_c0_config5(read_c0_config5() & ~MIPS_CONF5_FRE);
+ if (cpu_has_fre) {
+ /* clear FRE */
+ clear_c0_config5(MIPS_CONF5_FRE);
+ }
fr_common:
/* set CU1 & change FR appropriately */
fr = (int)mode & FPU_FR_MASK;
int ret = 0;
if (cpu_has_fpu) {
+ unsigned int config5;
+
ret = __own_fpu();
- if (!ret) {
- unsigned int config5 = read_c0_config5();
-
- /*
- * Ensure FRE is clear whilst running _init_fpu, since
- * single precision FP instructions are used. If FRE
- * was set then we'll just end up initialising all 32
- * 64b registers.
- */
- write_c0_config5(config5 & ~MIPS_CONF5_FRE);
- enable_fpu_hazard();
+ if (ret)
+ return ret;
+ if (!cpu_has_fre) {
_init_fpu();
- /* Restore FRE */
- write_c0_config5(config5);
- enable_fpu_hazard();
+ return 0;
}
+
+ /*
+ * Ensure FRE is clear whilst running _init_fpu, since
+ * single precision FP instructions are used. If FRE
+ * was set then we'll just end up initialising all 32
+ * 64b registers.
+ */
+ config5 = clear_c0_config5(MIPS_CONF5_FRE);
+ enable_fpu_hazard();
+
+ _init_fpu();
+
+ /* Restore FRE */
+ write_c0_config5(config5);
+ enable_fpu_hazard();
} else
fpu_emulator_init_fpu();
#define SGI_ARCS_REV 10 /* rev .10, 3/04/92 */
#endif
-typedef struct component {
+typedef struct {
CONFIGCLASS Class;
CONFIGTYPE Type;
IDENTIFIERFLAG Flags;
};
/* System ID */
-typedef struct systemid {
+typedef struct {
CHAR VendorId[8];
CHAR ProductId[8];
} SYSTEMID;
#endif /* _NT_PROM */
} MEMORYTYPE;
-typedef struct memorydescriptor {
+typedef struct {
MEMORYTYPE Type;
LONG BasePage;
LONG PageCount;
/* Macros to ease the creation of register access functions */
#define BUILD_CM_R_(name, off) \
-static inline u32 *addr_gcr_##name(void) \
+static inline u32 __iomem *addr_gcr_##name(void) \
{ \
- return (u32 *)(mips_cm_base + (off)); \
+ return (u32 __iomem *)(mips_cm_base + (off)); \
} \
\
static inline u32 read_gcr_##name(void) \
__res; \
})
+#define _write_32bit_cp1_register(dest, val, gas_hardfloat) \
+do { \
+ __asm__ __volatile__( \
+ " .set push \n" \
+ " .set reorder \n" \
+ " "STR(gas_hardfloat)" \n" \
+ " ctc1 %0,"STR(dest)" \n" \
+ " .set pop \n" \
+ : : "r" (val)); \
+} while (0)
+
#ifdef GAS_HAS_SET_HARDFLOAT
#define read_32bit_cp1_register(source) \
_read_32bit_cp1_register(source, .set hardfloat)
+#define write_32bit_cp1_register(dest, val) \
+ _write_32bit_cp1_register(dest, val, .set hardfloat)
#else
#define read_32bit_cp1_register(source) \
_read_32bit_cp1_register(source, )
+#define write_32bit_cp1_register(dest, val) \
+ _write_32bit_cp1_register(dest, val, )
#endif
#ifdef HAVE_AS_DSP
static inline long syscall_get_nr(struct task_struct *task,
struct pt_regs *regs)
{
- /* O32 ABI syscall() - Either 64-bit with O32 or 32-bit */
- if ((config_enabled(CONFIG_32BIT) ||
- test_tsk_thread_flag(task, TIF_32BIT_REGS)) &&
- (regs->regs[2] == __NR_syscall))
- return regs->regs[4];
- else
- return regs->regs[2];
+ return current_thread_info()->syscall;
}
static inline unsigned long mips_get_syscall_arg(unsigned long *arg,
*/
struct restart_block restart_block;
struct pt_regs *regs;
+ long syscall; /* syscall number */
};
/*
#define __NR_getrandom (__NR_Linux + 353)
#define __NR_memfd_create (__NR_Linux + 354)
#define __NR_bpf (__NR_Linux + 355)
+#define __NR_execveat (__NR_Linux + 356)
/*
* Offset of the last Linux o32 flavoured syscall
*/
-#define __NR_Linux_syscalls 355
+#define __NR_Linux_syscalls 356
#endif /* _MIPS_SIM == _MIPS_SIM_ABI32 */
#define __NR_O32_Linux 4000
-#define __NR_O32_Linux_syscalls 355
+#define __NR_O32_Linux_syscalls 356
#if _MIPS_SIM == _MIPS_SIM_ABI64
#define __NR_getrandom (__NR_Linux + 313)
#define __NR_memfd_create (__NR_Linux + 314)
#define __NR_bpf (__NR_Linux + 315)
+#define __NR_execveat (__NR_Linux + 316)
/*
* Offset of the last Linux 64-bit flavoured syscall
*/
-#define __NR_Linux_syscalls 315
+#define __NR_Linux_syscalls 316
#endif /* _MIPS_SIM == _MIPS_SIM_ABI64 */
#define __NR_64_Linux 5000
-#define __NR_64_Linux_syscalls 315
+#define __NR_64_Linux_syscalls 316
#if _MIPS_SIM == _MIPS_SIM_NABI32
#define __NR_getrandom (__NR_Linux + 317)
#define __NR_memfd_create (__NR_Linux + 318)
#define __NR_bpf (__NR_Linux + 319)
+#define __NR_execveat (__NR_Linux + 320)
/*
* Offset of the last N32 flavoured syscall
*/
-#define __NR_Linux_syscalls 319
+#define __NR_Linux_syscalls 320
#endif /* _MIPS_SIM == _MIPS_SIM_NABI32 */
#define __NR_N32_Linux 6000
-#define __NR_N32_Linux_syscalls 319
+#define __NR_N32_Linux_syscalls 320
#endif /* _UAPI_ASM_UNISTD_H */
#include <asm/irq_cpu.h>
#include <asm/mach-jz4740/base.h>
+#include <asm/mach-jz4740/irq.h>
+
+#include "irq.h"
static void __iomem *jz_intc_base;
int arch_elf_pt_proc(void *_ehdr, void *_phdr, struct file *elf,
bool is_interp, struct arch_elf_state *state)
{
- struct elfhdr *ehdr = _ehdr;
- struct elf_phdr *phdr = _phdr;
+ struct elf32_hdr *ehdr = _ehdr;
+ struct elf32_phdr *phdr = _phdr;
struct mips_elf_abiflags_v0 abiflags;
int ret;
return 0;
}
-static inline unsigned get_fp_abi(struct elfhdr *ehdr, int in_abi)
+static inline unsigned get_fp_abi(struct elf32_hdr *ehdr, int in_abi)
{
/* If the ABI requirement is provided, simply return that */
if (in_abi != -1)
int arch_check_elf(void *_ehdr, bool has_interpreter,
struct arch_elf_state *state)
{
- struct elfhdr *ehdr = _ehdr;
+ struct elf32_hdr *ehdr = _ehdr;
unsigned fp_abi, interp_fp_abi, abi0, abi1;
/* Ignore non-O32 binaries */
.irq_mask_ack = mask_mips_irq,
.irq_unmask = unmask_mips_irq,
.irq_eoi = unmask_mips_irq,
+ .irq_disable = mask_mips_irq,
+ .irq_enable = unmask_mips_irq,
};
/*
.irq_mask_ack = mips_mt_cpu_irq_ack,
.irq_unmask = unmask_mips_irq,
.irq_eoi = unmask_mips_irq,
+ .irq_disable = mask_mips_irq,
+ .irq_enable = unmask_mips_irq,
};
asmlinkage void __weak plat_irq_dispatch(void)
{
}
+int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
+{
+ /*
+ * Save any process state which is live in hardware registers to the
+ * parent context prior to duplication. This prevents the new child
+ * state becoming stale if the parent is preempted before copy_thread()
+ * gets a chance to save the parent's live hardware registers to the
+ * child context.
+ */
+ preempt_disable();
+
+ if (is_msa_enabled())
+ save_msa(current);
+ else if (is_fpu_owner())
+ _save_fp(current);
+
+ save_dsp(current);
+
+ preempt_enable();
+
+ *dst = *src;
+ return 0;
+}
+
int copy_thread(unsigned long clone_flags, unsigned long usp,
unsigned long arg, struct task_struct *p)
{
childksp = (unsigned long)task_stack_page(p) + THREAD_SIZE - 32;
- preempt_disable();
-
- if (is_msa_enabled())
- save_msa(p);
- else if (is_fpu_owner())
- save_fp(p);
-
- if (cpu_has_dsp)
- save_dsp(p);
-
- preempt_enable();
-
/* set up new TSS. */
childregs = (struct pt_regs *) childksp - 1;
/* Put the stack after the struct pt_regs. */
long ret = 0;
user_exit();
+ current_thread_info()->syscall = syscall;
+
if (secure_computing() == -1)
return -1;
sll t1, t0, 2
beqz v0, einval
lw t2, sys_call_table(t1) # syscall routine
+ sw a0, PT_R2(sp) # call routine directly on restart
/* Some syscalls like execve get their arguments from struct pt_regs
and claim zero arguments in the syscall table. Thus we have to
PTR sys_getrandom
PTR sys_memfd_create
PTR sys_bpf /* 4355 */
+ PTR sys_execveat
PTR sys_getrandom
PTR sys_memfd_create
PTR sys_bpf /* 5315 */
+ PTR sys_execveat
.size sys_call_table,.-sys_call_table
PTR sys_getrandom
PTR sys_memfd_create
PTR sys_bpf
+ PTR compat_sys_execveat /* 6320 */
.size sysn32_call_table,.-sysn32_call_table
dsll t1, t0, 3
beqz v0, einval
ld t2, sys32_call_table(t1) # syscall routine
+ sd a0, PT_R2(sp) # call routine directly on restart
move a0, a1 # shift argument registers
move a1, a2
PTR sys_getrandom
PTR sys_memfd_create
PTR sys_bpf /* 4355 */
+ PTR compat_sys_execveat
.size sys32_call_table,.-sys32_call_table
struct cpuinfo_mips *c __maybe_unused = ¤t_cpu_data;
/* Assume GIC is present */
- change_c0_status(ST0_IM, STATUSF_IP3 | STATUSF_IP4 | STATUSF_IP6 |
- STATUSF_IP7);
+ change_c0_status(ST0_IM, STATUSF_IP2 | STATUSF_IP3 | STATUSF_IP4 |
+ STATUSF_IP5 | STATUSF_IP6 | STATUSF_IP7);
/* Enable per-cpu interrupts: platform specific */
#ifdef CONFIG_MIPS_GIC
/* This is Malta specific: IPI,performance and timer interrupts */
if (gic_present)
- change_c0_status(ST0_IM, STATUSF_IP3 | STATUSF_IP4 |
+ change_c0_status(ST0_IM, STATUSF_IP2 | STATUSF_IP3 |
+ STATUSF_IP4 | STATUSF_IP5 |
STATUSF_IP6 | STATUSF_IP7);
else
#endif
unsigned int cpu;
cpu_probe();
- cpu_report();
per_cpu_trap_init(false);
mips_clockevent_init();
mp_ops->init_secondary();
+ cpu_report();
/*
* XXX parity protection should be folded in here when it's converted
/* Restore the scalar FP control & status register */
if (!was_fpu_owner)
- asm volatile("ctc1 %0, $31" : : "r"(current->thread.fpu.fcr31));
+ write_32bit_cp1_register(CP1_STATUS,
+ current->thread.fpu.fcr31);
}
out:
select PREEMPT_NOTIFIERS
select ANON_INODES
select KVM_MMIO
+ select SRCU
---help---
Support for hosting Guest kernels.
Currently supported on MIPS32 processors.
if (unlikely(fault & VM_FAULT_ERROR)) {
if (fault & VM_FAULT_OOM)
goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
else if (fault & VM_FAULT_SIGBUS)
goto do_sigbus;
BUG();
#ifdef CONFIG_64BIT
pg |= PG_ELPA;
#endif
+ if (cpu_has_rixiex)
+ pg |= PG_IEC;
write_c0_pagegrain(pg);
}
#define copy_from_user_page(vma, page, vaddr, dst, src, len) \
memcpy(dst, src, len)
-/*
- * Internal debugging function
- */
-#ifdef CONFIG_DEBUG_PAGEALLOC
-extern void kernel_map_pages(struct page *page, int numpages, int enable);
-#endif
-
#endif /* __ASSEMBLY__ */
#endif /* _ASM_CACHEFLUSH_H */
if (unlikely(fault & VM_FAULT_ERROR)) {
if (fault & VM_FAULT_OOM)
goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
else if (fault & VM_FAULT_SIGBUS)
goto do_sigbus;
BUG();
if (unlikely(fault & VM_FAULT_ERROR)) {
if (fault & VM_FAULT_OOM)
goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
else if (fault & VM_FAULT_SIGBUS)
goto do_sigbus;
BUG();
bad_area_nosemaphore:
/* User mode accesses just cause a SIGSEGV */
if (user_mode(regs)) {
- pr_alert("%s: unhandled page fault (%d) at 0x%08lx, "
- "cause %ld\n", current->comm, SIGSEGV, address, cause);
- show_regs(regs);
+ if (unhandled_signal(current, SIGSEGV) && printk_ratelimit()) {
+ pr_info("%s: unhandled page fault (%d) at 0x%08lx, "
+ "cause %ld\n", current->comm, SIGSEGV, address, cause);
+ show_regs(regs);
+ }
_exception(SIGSEGV, regs, code, address);
return;
}
if (unlikely(fault & VM_FAULT_ERROR)) {
if (fault & VM_FAULT_OOM)
goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
else if (fault & VM_FAULT_SIGBUS)
goto do_sigbus;
BUG();
*/
if (fault & VM_FAULT_OOM)
goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
else if (fault & VM_FAULT_SIGBUS)
goto bad_area;
BUG();
#define copy_from_user_page(vma, page, vaddr, dst, src, len) \
memcpy(dst, src, len)
-
-
-#ifdef CONFIG_DEBUG_PAGEALLOC
-/* internal debugging function */
-void kernel_map_pages(struct page *page, int numpages, int enable);
-#endif
-
#endif /* __KERNEL__ */
#endif /* _ASM_POWERPC_CACHEFLUSH_H */
select PREEMPT_NOTIFIERS
select ANON_INODES
select HAVE_KVM_EVENTFD
+ select SRCU
config KVM_BOOK3S_HANDLER
bool
if (*flt & VM_FAULT_OOM) {
ret = -ENOMEM;
goto out_unlock;
- } else if (*flt & VM_FAULT_SIGBUS) {
+ } else if (*flt & (VM_FAULT_SIGBUS | VM_FAULT_SIGSEGV)) {
ret = -EFAULT;
goto out_unlock;
}
*/
fault = handle_mm_fault(mm, vma, address, flags);
if (unlikely(fault & (VM_FAULT_RETRY|VM_FAULT_ERROR))) {
+ if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
rc = mm_fault_error(regs, address, fault);
if (rc >= MM_FAULT_RETURN)
goto bail;
/* Caches aren't brain-dead on the s390. */
#include <asm-generic/cacheflush.h>
-#ifdef CONFIG_DEBUG_PAGEALLOC
-void kernel_map_pages(struct page *page, int numpages, int enable);
-#endif
-
int set_memory_ro(unsigned long addr, int numpages);
int set_memory_rw(unsigned long addr, int numpages);
int set_memory_nx(unsigned long addr, int numpages);
select HAVE_KVM_IRQCHIP
select HAVE_KVM_IRQFD
select HAVE_KVM_IRQ_ROUTING
+ select SRCU
---help---
Support hosting paravirtualized guest machines using the SIE
virtualization capability on the mainframe. This should work
do_no_context(regs);
else
pagefault_out_of_memory();
+ } else if (fault & VM_FAULT_SIGSEGV) {
+ /* Kernel mode? Handle exceptions or die */
+ if (!user_mode(regs))
+ do_no_context(regs);
+ else
+ do_sigsegv(regs, SEGV_MAPERR);
} else if (fault & VM_FAULT_SIGBUS) {
/* Kernel mode? Handle exceptions or die */
if (!user_mode(regs))
if (unlikely(fault & VM_FAULT_ERROR)) {
if (fault & VM_FAULT_OOM)
goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
else if (fault & VM_FAULT_SIGBUS)
goto do_sigbus;
BUG();
} else {
if (fault & VM_FAULT_SIGBUS)
do_sigbus(regs, error_code, address);
+ else if (fault & VM_FAULT_SIGSEGV)
+ bad_area(regs, error_code, address);
else
BUG();
}
#define flush_cache_vmap(start, end) do { } while (0)
#define flush_cache_vunmap(start, end) do { } while (0)
-#ifdef CONFIG_DEBUG_PAGEALLOC
-/* internal debugging function */
-void kernel_map_pages(struct page *page, int numpages, int enable);
-#endif
-
#endif /* !__ASSEMBLY__ */
#endif /* _SPARC64_CACHEFLUSH_H */
if (unlikely(fault & VM_FAULT_ERROR)) {
if (fault & VM_FAULT_OOM)
goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
else if (fault & VM_FAULT_SIGBUS)
goto do_sigbus;
BUG();
if (unlikely(fault & VM_FAULT_ERROR)) {
if (fault & VM_FAULT_OOM)
goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
else if (fault & VM_FAULT_SIGBUS)
goto do_sigbus;
BUG();
depends on HAVE_KVM && MODULES
select PREEMPT_NOTIFIERS
select ANON_INODES
+ select SRCU
---help---
Support hosting paravirtualized guest machines.
if (unlikely(fault & VM_FAULT_ERROR)) {
if (fault & VM_FAULT_OOM)
goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
else if (fault & VM_FAULT_SIGBUS)
goto do_sigbus;
BUG();
if (unlikely(fault & VM_FAULT_ERROR)) {
if (fault & VM_FAULT_OOM) {
goto out_of_memory;
+ } else if (fault & VM_FAULT_SIGSEGV) {
+ goto out;
} else if (fault & VM_FAULT_SIGBUS) {
err = -EACCES;
goto out;
select HAVE_ACPI_APEI_NMI if ACPI
select ACPI_LEGACY_TABLES_LOOKUP if ACPI
select X86_FEATURE_NAMES if PROC_FS
+ select SRCU
config INSTRUCTION_DECODER
def_bool y
source "kernel/Kconfig.preempt"
+config UP_LATE_INIT
+ def_bool y
+ depends on !SMP && X86_LOCAL_APIC
+
config X86_UP_APIC
bool "Local APIC support on uniprocessors"
depends on X86_32 && !SMP && !X86_32_NON_STANDARD
-#ifndef BOOT_ISDIGIT_H
-
-#define BOOT_ISDIGIT_H
+#ifndef BOOT_CTYPE_H
+#define BOOT_CTYPE_H
static inline int isdigit(int ch)
{
#define DEFAULT_SERIAL_PORT 0x3f8 /* ttyS0 */
-#define XMTRDY 0x20
-
#define DLAB 0x80
#define TXR 0 /* Transmit register (WRITE) */
static const int bases[] = { 0x3f8, 0x2f8 };
int idx = 0;
- if (!strncmp(arg + pos, "ttyS", 4))
- pos += 4;
+ /* += strlen("ttyS"); */
+ pos += 4;
if (arg[pos++] == '1')
idx = 1;
sysexit_from_sys_call:
andl $~TS_COMPAT,TI_status+THREAD_INFO(%rsp,RIP-ARGOFFSET)
/* clear IF, that popfq doesn't enable interrupts early */
- andl $~0x200,EFLAGS-R11(%rsp)
- movl RIP-R11(%rsp),%edx /* User %eip */
+ andl $~0x200,EFLAGS-ARGOFFSET(%rsp)
+ movl RIP-ARGOFFSET(%rsp),%edx /* User %eip */
CFI_REGISTER rip,rdx
RESTORE_ARGS 0,24,0,0,0,0
xorq %r8,%r8
extern void native_apic_icr_write(u32 low, u32 id);
extern u64 native_apic_icr_read(void);
-extern int x2apic_mode;
+static inline bool apic_is_x2apic_enabled(void)
+{
+ u64 msr;
+
+ if (rdmsrl_safe(MSR_IA32_APICBASE, &msr))
+ return false;
+ return msr & X2APIC_ENABLE;
+}
#ifdef CONFIG_X86_X2APIC
/*
return val;
}
+extern int x2apic_mode;
extern int x2apic_phys;
-extern int x2apic_preenabled;
-extern void check_x2apic(void);
-extern void enable_x2apic(void);
+extern void __init check_x2apic(void);
+extern void x2apic_setup(void);
static inline int x2apic_enabled(void)
{
- u64 msr;
-
- if (!cpu_has_x2apic)
- return 0;
-
- rdmsrl(MSR_IA32_APICBASE, msr);
- if (msr & X2APIC_ENABLE)
- return 1;
- return 0;
+ return cpu_has_x2apic && apic_is_x2apic_enabled();
}
#define x2apic_supported() (cpu_has_x2apic)
-static inline void x2apic_force_phys(void)
-{
- x2apic_phys = 1;
-}
#else
-static inline void disable_x2apic(void)
-{
-}
-static inline void check_x2apic(void)
-{
-}
-static inline void enable_x2apic(void)
-{
-}
-static inline int x2apic_enabled(void)
-{
- return 0;
-}
-static inline void x2apic_force_phys(void)
-{
-}
+static inline void check_x2apic(void) { }
+static inline void x2apic_setup(void) { }
+static inline int x2apic_enabled(void) { return 0; }
-#define x2apic_preenabled 0
-#define x2apic_supported() 0
+#define x2apic_mode (0)
+#define x2apic_supported() (0)
#endif
extern void enable_IR_x2apic(void);
extern int lapic_get_maxlvt(void);
extern void clear_local_APIC(void);
-extern void connect_bsp_APIC(void);
extern void disconnect_bsp_APIC(int virt_wire_setup);
extern void disable_local_APIC(void);
extern void lapic_shutdown(void);
extern void sync_Arb_IDs(void);
extern void init_bsp_APIC(void);
extern void setup_local_APIC(void);
-extern void end_local_APIC_setup(void);
-extern void bsp_end_local_APIC_setup(void);
extern void init_apic_mappings(void);
void register_lapic_address(unsigned long address);
extern void setup_boot_APIC_clock(void);
extern int APIC_init_uniprocessor(void);
extern int apic_force_enable(unsigned long addr);
+extern int apic_bsp_setup(bool upmode);
+extern void apic_ap_setup(void);
+
/*
* On 32bit this is mach-xxx local
*/
#define SS 160
#define ARGOFFSET R11
-#define SWFRAME ORIG_RAX
.macro SAVE_ARGS addskip=0, save_rcx=1, save_r891011=1, rax_enosys=0
subq $9*8+\addskip, %rsp
#define X86_FEATURE_TOPOEXT ( 6*32+22) /* topology extensions CPUID leafs */
#define X86_FEATURE_PERFCTR_CORE ( 6*32+23) /* core performance counter extensions */
#define X86_FEATURE_PERFCTR_NB ( 6*32+24) /* NB performance counter extensions */
+#define X86_FEATURE_BPEXT (6*32+26) /* data breakpoint extension */
#define X86_FEATURE_PERFCTR_L2 ( 6*32+28) /* L2 performance counter extensions */
/*
#define cpu_has_cx16 boot_cpu_has(X86_FEATURE_CX16)
#define cpu_has_eager_fpu boot_cpu_has(X86_FEATURE_EAGER_FPU)
#define cpu_has_topoext boot_cpu_has(X86_FEATURE_TOPOEXT)
+#define cpu_has_bpext boot_cpu_has(X86_FEATURE_BPEXT)
#if __GNUC__ >= 4
extern void warn_pre_alternatives(void);
static inline void debug_stack_usage_dec(void) { }
#endif /* X86_64 */
+#ifdef CONFIG_CPU_SUP_AMD
+extern void set_dr_addr_mask(unsigned long mask, int dr);
+#else
+static inline void set_dr_addr_mask(unsigned long mask, int dr) { }
+#endif
#endif /* _ASM_X86_DEBUGREG_H */
*/
struct arch_hw_breakpoint {
unsigned long address;
+ unsigned long mask;
u8 len;
u8 type;
};
#define native_ioapic_set_affinity NULL
#define native_setup_ioapic_entry NULL
#define native_eoi_ioapic_pin NULL
+
+static inline void setup_IO_APIC(void) { }
+static inline void enable_IO_APIC(void) { }
+static inline void setup_ioapic_dest(void) { }
+
#endif
#endif /* _ASM_X86_IO_APIC_H */
#ifdef CONFIG_IRQ_REMAP
-extern void setup_irq_remapping_ops(void);
-extern int irq_remapping_supported(void);
extern void set_irq_remapping_broken(void);
extern int irq_remapping_prepare(void);
extern int irq_remapping_enable(void);
#else /* CONFIG_IRQ_REMAP */
-static inline void setup_irq_remapping_ops(void) { }
-static inline int irq_remapping_supported(void) { return 0; }
static inline void set_irq_remapping_broken(void) { }
static inline int irq_remapping_prepare(void) { return -ENODEV; }
static inline int irq_remapping_enable(void) { return -ENODEV; }
void machine_check_poll(enum mcp_flags flags, mce_banks_t *b);
int mce_notify_irq(void);
-void mce_notify_process(void);
DECLARE_PER_CPU(struct mce, injectm);
+++ /dev/null
-/* two abstractions specific to kernel/smpboot.c, mainly to cater to visws
- * which needs to alter them. */
-
-static inline void smpboot_clear_io_apic_irqs(void)
-{
-#ifdef CONFIG_X86_IO_APIC
- io_apic_irqs = 0;
-#endif
-}
-
-static inline void smpboot_setup_warm_reset_vector(unsigned long start_eip)
-{
- unsigned long flags;
-
- spin_lock_irqsave(&rtc_lock, flags);
- CMOS_WRITE(0xa, 0xf);
- spin_unlock_irqrestore(&rtc_lock, flags);
- local_flush_tlb();
- pr_debug("1.\n");
- *((volatile unsigned short *)phys_to_virt(TRAMPOLINE_PHYS_HIGH)) =
- start_eip >> 4;
- pr_debug("2.\n");
- *((volatile unsigned short *)phys_to_virt(TRAMPOLINE_PHYS_LOW)) =
- start_eip & 0xf;
- pr_debug("3.\n");
-}
-
-static inline void smpboot_restore_warm_reset_vector(void)
-{
- unsigned long flags;
-
- /*
- * Install writable page 0 entry to set BIOS data area.
- */
- local_flush_tlb();
-
- /*
- * Paranoid: Set warm reset code and vector here back
- * to default values.
- */
- spin_lock_irqsave(&rtc_lock, flags);
- CMOS_WRITE(0, 0xf);
- spin_unlock_irqrestore(&rtc_lock, flags);
-
- *((volatile u32 *)phys_to_virt(TRAMPOLINE_PHYS_LOW)) = 0;
-}
-
-static inline void __init smpboot_setup_io_apic(void)
-{
-#ifdef CONFIG_X86_IO_APIC
- /*
- * Here we can be sure that there is an IO-APIC in the system. Let's
- * go and set it up:
- */
- if (!skip_ioapic_setup && nr_ioapics)
- setup_IO_APIC();
- else {
- nr_ioapics = 0;
- }
-#endif
-}
-
-static inline void smpboot_clear_io_apic(void)
-{
-#ifdef CONFIG_X86_IO_APIC
- nr_ioapics = 0;
-#endif
-}
#define TIF_SYSCALL_EMU 6 /* syscall emulation active */
#define TIF_SYSCALL_AUDIT 7 /* syscall auditing active */
#define TIF_SECCOMP 8 /* secure computing */
-#define TIF_MCE_NOTIFY 10 /* notify userspace of an MCE */
#define TIF_USER_RETURN_NOTIFY 11 /* notify kernel of userspace return */
#define TIF_UPROBE 12 /* breakpointed or singlestepping */
#define TIF_NOTSC 16 /* TSC is not accessible in userland */
#define _TIF_SYSCALL_EMU (1 << TIF_SYSCALL_EMU)
#define _TIF_SYSCALL_AUDIT (1 << TIF_SYSCALL_AUDIT)
#define _TIF_SECCOMP (1 << TIF_SECCOMP)
-#define _TIF_MCE_NOTIFY (1 << TIF_MCE_NOTIFY)
#define _TIF_USER_RETURN_NOTIFY (1 << TIF_USER_RETURN_NOTIFY)
#define _TIF_UPROBE (1 << TIF_UPROBE)
#define _TIF_NOTSC (1 << TIF_NOTSC)
/* Only used for 64 bit */
#define _TIF_DO_NOTIFY_MASK \
- (_TIF_SIGPENDING | _TIF_MCE_NOTIFY | _TIF_NOTIFY_RESUME | \
+ (_TIF_SIGPENDING | _TIF_NOTIFY_RESUME | \
_TIF_USER_RETURN_NOTIFY | _TIF_UPROBE)
/* flags to check in __switch_to() */
return ti;
}
+static inline unsigned long current_stack_pointer(void)
+{
+ unsigned long sp;
+#ifdef CONFIG_X86_64
+ asm("mov %%rsp,%0" : "=g" (sp));
+#else
+ asm("mov %%esp,%0" : "=g" (sp));
+#endif
+ return sp;
+}
+
#else /* !__ASSEMBLY__ */
/* how to get the thread information struct from ASM */
#ifndef _ASM_X86_TRAPS_H
#define _ASM_X86_TRAPS_H
+#include <linux/context_tracking_state.h>
#include <linux/kprobes.h>
#include <asm/debugreg.h>
asmlinkage void mce_threshold_interrupt(void);
#endif
+extern enum ctx_state ist_enter(struct pt_regs *regs);
+extern void ist_exit(struct pt_regs *regs, enum ctx_state prev_state);
+extern void ist_begin_non_atomic(struct pt_regs *regs);
+extern void ist_end_non_atomic(void);
+
/* Interrupts/Exceptions */
enum {
X86_TRAP_DE = 0, /* 0, Divide-by-zero */
/* Fam 16h MSRs */
#define MSR_F16H_L2I_PERF_CTL 0xc0010230
#define MSR_F16H_L2I_PERF_CTR 0xc0010231
+#define MSR_F16H_DR1_ADDR_MASK 0xc0011019
+#define MSR_F16H_DR2_ADDR_MASK 0xc001101a
+#define MSR_F16H_DR3_ADDR_MASK 0xc001101b
+#define MSR_F16H_DR0_ADDR_MASK 0xc0011027
/* Fam 15h MSRs */
#define MSR_F15H_PERF_CTL 0xc0010200
return gsi;
}
+#ifdef CONFIG_X86_LOCAL_APIC
static int acpi_register_gsi_ioapic(struct device *dev, u32 gsi,
int trigger, int polarity)
{
mutex_unlock(&acpi_ioapic_lock);
#endif
}
+#endif
int (*__acpi_register_gsi)(struct device *dev, u32 gsi,
int trigger, int polarity) = acpi_register_gsi_pic;
*/
static int force_enable_local_apic __initdata;
-/* Control whether x2APIC mode is enabled or not */
-static bool nox2apic __initdata;
-
/*
* APIC command line parameters
*/
__setup("apicpmtimer", setup_apicpmtimer);
#endif
-int x2apic_mode;
-#ifdef CONFIG_X86_X2APIC
-/* x2apic enabled before OS handover */
-int x2apic_preenabled;
-static int x2apic_disabled;
-static int __init setup_nox2apic(char *str)
-{
- if (x2apic_enabled()) {
- int apicid = native_apic_msr_read(APIC_ID);
-
- if (apicid >= 255) {
- pr_warning("Apicid: %08x, cannot enforce nox2apic\n",
- apicid);
- return 0;
- }
-
- pr_warning("x2apic already enabled. will disable it\n");
- } else
- setup_clear_cpu_cap(X86_FEATURE_X2APIC);
-
- nox2apic = true;
-
- return 0;
-}
-early_param("nox2apic", setup_nox2apic);
-#endif
-
unsigned long mp_lapic_addr;
int disable_apic;
/* Disable local APIC timer from the kernel commandline or via dmi quirk */
#endif
}
-void end_local_APIC_setup(void)
+static void end_local_APIC_setup(void)
{
lapic_setup_esr();
apic_pm_activate();
}
-void __init bsp_end_local_APIC_setup(void)
+/*
+ * APIC setup function for application processors. Called from smpboot.c
+ */
+void apic_ap_setup(void)
{
+ setup_local_APIC();
end_local_APIC_setup();
-
- /*
- * Now that local APIC setup is completed for BP, configure the fault
- * handling for interrupt remapping.
- */
- irq_remap_enable_fault_handling();
-
}
#ifdef CONFIG_X86_X2APIC
-/*
- * Need to disable xapic and x2apic at the same time and then enable xapic mode
- */
-static inline void __disable_x2apic(u64 msr)
-{
- wrmsrl(MSR_IA32_APICBASE,
- msr & ~(X2APIC_ENABLE | XAPIC_ENABLE));
- wrmsrl(MSR_IA32_APICBASE, msr & ~X2APIC_ENABLE);
-}
+int x2apic_mode;
-static __init void disable_x2apic(void)
+enum {
+ X2APIC_OFF,
+ X2APIC_ON,
+ X2APIC_DISABLED,
+};
+static int x2apic_state;
+
+static inline void __x2apic_disable(void)
{
u64 msr;
- if (!cpu_has_x2apic)
+ if (cpu_has_apic)
return;
rdmsrl(MSR_IA32_APICBASE, msr);
- if (msr & X2APIC_ENABLE) {
- u32 x2apic_id = read_apic_id();
-
- if (x2apic_id >= 255)
- panic("Cannot disable x2apic, id: %08x\n", x2apic_id);
+ if (!(msr & X2APIC_ENABLE))
+ return;
+ /* Disable xapic and x2apic first and then reenable xapic mode */
+ wrmsrl(MSR_IA32_APICBASE, msr & ~(X2APIC_ENABLE | XAPIC_ENABLE));
+ wrmsrl(MSR_IA32_APICBASE, msr & ~X2APIC_ENABLE);
+ printk_once(KERN_INFO "x2apic disabled\n");
+}
- pr_info("Disabling x2apic\n");
- __disable_x2apic(msr);
+static inline void __x2apic_enable(void)
+{
+ u64 msr;
- if (nox2apic) {
- clear_cpu_cap(&cpu_data(0), X86_FEATURE_X2APIC);
- setup_clear_cpu_cap(X86_FEATURE_X2APIC);
- }
+ rdmsrl(MSR_IA32_APICBASE, msr);
+ if (msr & X2APIC_ENABLE)
+ return;
+ wrmsrl(MSR_IA32_APICBASE, msr | X2APIC_ENABLE);
+ printk_once(KERN_INFO "x2apic enabled\n");
+}
- x2apic_disabled = 1;
- x2apic_mode = 0;
+static int __init setup_nox2apic(char *str)
+{
+ if (x2apic_enabled()) {
+ int apicid = native_apic_msr_read(APIC_ID);
- register_lapic_address(mp_lapic_addr);
+ if (apicid >= 255) {
+ pr_warning("Apicid: %08x, cannot enforce nox2apic\n",
+ apicid);
+ return 0;
+ }
+ pr_warning("x2apic already enabled.\n");
+ __x2apic_disable();
}
+ setup_clear_cpu_cap(X86_FEATURE_X2APIC);
+ x2apic_state = X2APIC_DISABLED;
+ x2apic_mode = 0;
+ return 0;
}
+early_param("nox2apic", setup_nox2apic);
-void check_x2apic(void)
+/* Called from cpu_init() to enable x2apic on (secondary) cpus */
+void x2apic_setup(void)
{
- if (x2apic_enabled()) {
- pr_info("x2apic enabled by BIOS, switching to x2apic ops\n");
- x2apic_preenabled = x2apic_mode = 1;
+ /*
+ * If x2apic is not in ON state, disable it if already enabled
+ * from BIOS.
+ */
+ if (x2apic_state != X2APIC_ON) {
+ __x2apic_disable();
+ return;
}
+ __x2apic_enable();
}
-void enable_x2apic(void)
+static __init void x2apic_disable(void)
{
- u64 msr;
+ u32 x2apic_id;
- rdmsrl(MSR_IA32_APICBASE, msr);
- if (x2apic_disabled) {
- __disable_x2apic(msr);
+ if (x2apic_state != X2APIC_ON)
+ goto out;
+
+ x2apic_id = read_apic_id();
+ if (x2apic_id >= 255)
+ panic("Cannot disable x2apic, id: %08x\n", x2apic_id);
+
+ __x2apic_disable();
+ register_lapic_address(mp_lapic_addr);
+out:
+ x2apic_state = X2APIC_DISABLED;
+ x2apic_mode = 0;
+}
+
+static __init void x2apic_enable(void)
+{
+ if (x2apic_state != X2APIC_OFF)
return;
- }
- if (!x2apic_mode)
+ x2apic_mode = 1;
+ x2apic_state = X2APIC_ON;
+ __x2apic_enable();
+}
+
+static __init void try_to_enable_x2apic(int remap_mode)
+{
+ if (x2apic_state == X2APIC_DISABLED)
return;
- if (!(msr & X2APIC_ENABLE)) {
- printk_once(KERN_INFO "Enabling x2apic\n");
- wrmsrl(MSR_IA32_APICBASE, msr | X2APIC_ENABLE);
+ if (remap_mode != IRQ_REMAP_X2APIC_MODE) {
+ /* IR is required if there is APIC ID > 255 even when running
+ * under KVM
+ */
+ if (max_physical_apicid > 255 ||
+ (IS_ENABLED(CONFIG_HYPERVISOR_GUEST) &&
+ !hypervisor_x2apic_available())) {
+ pr_info("x2apic: IRQ remapping doesn't support X2APIC mode\n");
+ x2apic_disable();
+ return;
+ }
+
+ /*
+ * without IR all CPUs can be addressed by IOAPIC/MSI
+ * only in physical mode
+ */
+ x2apic_phys = 1;
}
+ x2apic_enable();
}
-#endif /* CONFIG_X86_X2APIC */
-int __init enable_IR(void)
+void __init check_x2apic(void)
{
-#ifdef CONFIG_IRQ_REMAP
- if (!irq_remapping_supported()) {
- pr_debug("intr-remapping not supported\n");
- return -1;
+ if (x2apic_enabled()) {
+ pr_info("x2apic: enabled by BIOS, switching to x2apic ops\n");
+ x2apic_mode = 1;
+ x2apic_state = X2APIC_ON;
+ } else if (!cpu_has_x2apic) {
+ x2apic_state = X2APIC_DISABLED;
}
+}
+#else /* CONFIG_X86_X2APIC */
+static int __init validate_x2apic(void)
+{
+ if (!apic_is_x2apic_enabled())
+ return 0;
+ /*
+ * Checkme: Can we simply turn off x2apic here instead of panic?
+ */
+ panic("BIOS has enabled x2apic but kernel doesn't support x2apic, please disable x2apic in BIOS.\n");
+}
+early_initcall(validate_x2apic);
- if (!x2apic_preenabled && skip_ioapic_setup) {
- pr_info("Skipped enabling intr-remap because of skipping "
- "io-apic setup\n");
+static inline void try_to_enable_x2apic(int remap_mode) { }
+static inline void __x2apic_enable(void) { }
+#endif /* !CONFIG_X86_X2APIC */
+
+static int __init try_to_enable_IR(void)
+{
+#ifdef CONFIG_X86_IO_APIC
+ if (!x2apic_enabled() && skip_ioapic_setup) {
+ pr_info("Not enabling interrupt remapping due to skipped IO-APIC setup\n");
return -1;
}
-
- return irq_remapping_enable();
#endif
- return -1;
+ return irq_remapping_enable();
}
void __init enable_IR_x2apic(void)
{
unsigned long flags;
- int ret, x2apic_enabled = 0;
- int hardware_init_ret;
-
- /* Make sure irq_remap_ops are initialized */
- setup_irq_remapping_ops();
+ int ret, ir_stat;
- hardware_init_ret = irq_remapping_prepare();
- if (hardware_init_ret && !x2apic_supported())
+ ir_stat = irq_remapping_prepare();
+ if (ir_stat < 0 && !x2apic_supported())
return;
ret = save_ioapic_entries();
legacy_pic->mask_all();
mask_ioapic_entries();
- if (x2apic_preenabled && nox2apic)
- disable_x2apic();
-
- if (hardware_init_ret)
- ret = -1;
- else
- ret = enable_IR();
-
- if (!x2apic_supported())
- goto skip_x2apic;
+ /* If irq_remapping_prepare() succeded, try to enable it */
+ if (ir_stat >= 0)
+ ir_stat = try_to_enable_IR();
+ /* ir_stat contains the remap mode or an error code */
+ try_to_enable_x2apic(ir_stat);
- if (ret < 0) {
- /* IR is required if there is APIC ID > 255 even when running
- * under KVM
- */
- if (max_physical_apicid > 255 ||
- !hypervisor_x2apic_available()) {
- if (x2apic_preenabled)
- disable_x2apic();
- goto skip_x2apic;
- }
- /*
- * without IR all CPUs can be addressed by IOAPIC/MSI
- * only in physical mode
- */
- x2apic_force_phys();
- }
-
- if (ret == IRQ_REMAP_XAPIC_MODE) {
- pr_info("x2apic not enabled, IRQ remapping is in xapic mode\n");
- goto skip_x2apic;
- }
-
- x2apic_enabled = 1;
-
- if (x2apic_supported() && !x2apic_mode) {
- x2apic_mode = 1;
- enable_x2apic();
- pr_info("Enabled x2apic\n");
- }
-
-skip_x2apic:
- if (ret < 0) /* IR enabling failed */
+ if (ir_stat < 0)
restore_ioapic_entries();
legacy_pic->restore_mask();
local_irq_restore(flags);
}
}
-/*
- * This initializes the IO-APIC and APIC hardware if this is
- * a UP kernel.
- */
int apic_version[MAX_LOCAL_APIC];
-int __init APIC_init_uniprocessor(void)
-{
- if (disable_apic) {
- pr_info("Apic disabled\n");
- return -1;
- }
-#ifdef CONFIG_X86_64
- if (!cpu_has_apic) {
- disable_apic = 1;
- pr_info("Apic disabled by BIOS\n");
- return -1;
- }
-#else
- if (!smp_found_config && !cpu_has_apic)
- return -1;
-
- /*
- * Complain if the BIOS pretends there is one.
- */
- if (!cpu_has_apic &&
- APIC_INTEGRATED(apic_version[boot_cpu_physical_apicid])) {
- pr_err("BIOS bug, local APIC 0x%x not detected!...\n",
- boot_cpu_physical_apicid);
- return -1;
- }
-#endif
-
- default_setup_apic_routing();
-
- verify_local_APIC();
- connect_bsp_APIC();
-
-#ifdef CONFIG_X86_64
- apic_write(APIC_ID, SET_APIC_ID(boot_cpu_physical_apicid));
-#else
- /*
- * Hack: In case of kdump, after a crash, kernel might be booting
- * on a cpu with non-zero lapic id. But boot_cpu_physical_apicid
- * might be zero if read from MP tables. Get it from LAPIC.
- */
-# ifdef CONFIG_CRASH_DUMP
- boot_cpu_physical_apicid = read_apic_id();
-# endif
-#endif
- physid_set_mask_of_physid(boot_cpu_physical_apicid, &phys_cpu_present_map);
- setup_local_APIC();
-
-#ifdef CONFIG_X86_IO_APIC
- /*
- * Now enable IO-APICs, actually call clear_IO_APIC
- * We need clear_IO_APIC before enabling error vector
- */
- if (!skip_ioapic_setup && nr_ioapics)
- enable_IO_APIC();
-#endif
-
- bsp_end_local_APIC_setup();
-
-#ifdef CONFIG_X86_IO_APIC
- if (smp_found_config && !skip_ioapic_setup && nr_ioapics)
- setup_IO_APIC();
- else {
- nr_ioapics = 0;
- }
-#endif
-
- x86_init.timers.setup_percpu_clockev();
- return 0;
-}
-
/*
* Local APIC interrupts
*/
/**
* connect_bsp_APIC - attach the APIC to the interrupt system
*/
-void __init connect_bsp_APIC(void)
+static void __init connect_bsp_APIC(void)
{
#ifdef CONFIG_X86_32
if (pic_mode) {
}
}
+static void __init apic_bsp_up_setup(void)
+{
+#ifdef CONFIG_X86_64
+ apic_write(APIC_ID, SET_APIC_ID(boot_cpu_physical_apicid));
+#else
+ /*
+ * Hack: In case of kdump, after a crash, kernel might be booting
+ * on a cpu with non-zero lapic id. But boot_cpu_physical_apicid
+ * might be zero if read from MP tables. Get it from LAPIC.
+ */
+# ifdef CONFIG_CRASH_DUMP
+ boot_cpu_physical_apicid = read_apic_id();
+# endif
+#endif
+ physid_set_mask_of_physid(boot_cpu_physical_apicid, &phys_cpu_present_map);
+}
+
+/**
+ * apic_bsp_setup - Setup function for local apic and io-apic
+ * @upmode: Force UP mode (for APIC_init_uniprocessor)
+ *
+ * Returns:
+ * apic_id of BSP APIC
+ */
+int __init apic_bsp_setup(bool upmode)
+{
+ int id;
+
+ connect_bsp_APIC();
+ if (upmode)
+ apic_bsp_up_setup();
+ setup_local_APIC();
+
+ if (x2apic_mode)
+ id = apic_read(APIC_LDR);
+ else
+ id = GET_APIC_LOGICAL_ID(apic_read(APIC_LDR));
+
+ enable_IO_APIC();
+ end_local_APIC_setup();
+ irq_remap_enable_fault_handling();
+ setup_IO_APIC();
+ /* Setup local timer */
+ x86_init.timers.setup_percpu_clockev();
+ return id;
+}
+
+/*
+ * This initializes the IO-APIC and APIC hardware if this is
+ * a UP kernel.
+ */
+int __init APIC_init_uniprocessor(void)
+{
+ if (disable_apic) {
+ pr_info("Apic disabled\n");
+ return -1;
+ }
+#ifdef CONFIG_X86_64
+ if (!cpu_has_apic) {
+ disable_apic = 1;
+ pr_info("Apic disabled by BIOS\n");
+ return -1;
+ }
+#else
+ if (!smp_found_config && !cpu_has_apic)
+ return -1;
+
+ /*
+ * Complain if the BIOS pretends there is one.
+ */
+ if (!cpu_has_apic &&
+ APIC_INTEGRATED(apic_version[boot_cpu_physical_apicid])) {
+ pr_err("BIOS bug, local APIC 0x%x not detected!...\n",
+ boot_cpu_physical_apicid);
+ return -1;
+ }
+#endif
+
+ if (!smp_found_config)
+ disable_ioapic_support();
+
+ default_setup_apic_routing();
+ verify_local_APIC();
+ apic_bsp_setup(true);
+ return 0;
+}
+
+#ifdef CONFIG_UP_LATE_INIT
+void __init up_late_init(void)
+{
+ APIC_init_uniprocessor();
+}
+#endif
+
/*
* Power management
*/
mask_ioapic_entries();
legacy_pic->mask_all();
- if (x2apic_mode)
- enable_x2apic();
- else {
+ if (x2apic_mode) {
+ __x2apic_enable();
+ } else {
/*
* Make sure the APICBASE points to the right address
*
int i8259_apic, i8259_pin;
int apic, pin;
- if (!nr_legacy_irqs())
+ if (skip_ioapic_setup)
+ nr_ioapics = 0;
+
+ if (!nr_legacy_irqs() || !nr_ioapics)
return;
for_each_ioapic_pin(apic, pin) {
}
local_irq_disable();
apic_printk(APIC_QUIET, KERN_INFO "..... failed :(.\n");
- if (x2apic_preenabled)
+ if (apic_is_x2apic_enabled())
apic_printk(APIC_QUIET, KERN_INFO
"Perhaps problem with the pre-enabled x2apic mode\n"
"Try booting with x2apic and interrupt-remapping disabled in the bios.\n");
{
int ioapic;
- /*
- * calling enable_IO_APIC() is moved to setup_local_APIC for BP
- */
+ if (skip_ioapic_setup || !nr_ioapics)
+ return;
+
io_apic_irqs = nr_legacy_irqs() ? ~PIC_IRQS : ~0UL;
apic_printk(APIC_VERBOSE, "ENABLING IO-APIC IRQs\n");
return false;
}
+
+void set_dr_addr_mask(unsigned long mask, int dr)
+{
+ if (!cpu_has_bpext)
+ return;
+
+ switch (dr) {
+ case 0:
+ wrmsr(MSR_F16H_DR0_ADDR_MASK, mask, 0);
+ break;
+ case 1:
+ case 2:
+ case 3:
+ wrmsr(MSR_F16H_DR1_ADDR_MASK - 1 + dr, mask, 0);
+ break;
+ default:
+ break;
+ }
+}
u16 __read_mostly tlb_lld_4m[NR_INFO];
u16 __read_mostly tlb_lld_1g[NR_INFO];
-void cpu_detect_tlb(struct cpuinfo_x86 *c)
+static void cpu_detect_tlb(struct cpuinfo_x86 *c)
{
if (this_cpu->c_detect_tlb)
this_cpu->c_detect_tlb(c);
- printk(KERN_INFO "Last level iTLB entries: 4KB %d, 2MB %d, 4MB %d\n"
- "Last level dTLB entries: 4KB %d, 2MB %d, 4MB %d, 1GB %d\n",
+ pr_info("Last level iTLB entries: 4KB %d, 2MB %d, 4MB %d\n",
tlb_lli_4k[ENTRIES], tlb_lli_2m[ENTRIES],
- tlb_lli_4m[ENTRIES], tlb_lld_4k[ENTRIES],
- tlb_lld_2m[ENTRIES], tlb_lld_4m[ENTRIES],
- tlb_lld_1g[ENTRIES]);
+ tlb_lli_4m[ENTRIES]);
+
+ pr_info("Last level dTLB entries: 4KB %d, 2MB %d, 4MB %d, 1GB %d\n",
+ tlb_lld_4k[ENTRIES], tlb_lld_2m[ENTRIES],
+ tlb_lld_4m[ENTRIES], tlb_lld_1g[ENTRIES]);
}
void detect_ht(struct cpuinfo_x86 *c)
barrier();
x86_configure_nx();
- enable_x2apic();
+ x2apic_setup();
/*
* set up and load the per-CPU TSS
rdmsrl(MSR_IA32_ENERGY_PERF_BIAS, epb);
if ((epb & 0xF) == ENERGY_PERF_BIAS_PERFORMANCE) {
- printk_once(KERN_WARNING "ENERGY_PERF_BIAS:"
- " Set to 'normal', was 'performance'\n"
- "ENERGY_PERF_BIAS: View and update with"
- " x86_energy_perf_policy(8)\n");
+ pr_warn_once("ENERGY_PERF_BIAS: Set to 'normal', was 'performance'\n");
+ pr_warn_once("ENERGY_PERF_BIAS: View and update with x86_energy_perf_policy(8)\n");
epb = (epb & ~0xF) | ENERGY_PERF_BIAS_NORMAL;
wrmsrl(MSR_IA32_ENERGY_PERF_BIAS, epb);
}
#include <linux/export.h>
#include <asm/processor.h>
+#include <asm/traps.h>
#include <asm/mce.h>
#include <asm/msr.h>
}
}
-/*
- * Need to save faulting physical address associated with a process
- * in the machine check handler some place where we can grab it back
- * later in mce_notify_process()
- */
-#define MCE_INFO_MAX 16
-
-struct mce_info {
- atomic_t inuse;
- struct task_struct *t;
- __u64 paddr;
- int restartable;
-} mce_info[MCE_INFO_MAX];
-
-static void mce_save_info(__u64 addr, int c)
-{
- struct mce_info *mi;
-
- for (mi = mce_info; mi < &mce_info[MCE_INFO_MAX]; mi++) {
- if (atomic_cmpxchg(&mi->inuse, 0, 1) == 0) {
- mi->t = current;
- mi->paddr = addr;
- mi->restartable = c;
- return;
- }
- }
-
- mce_panic("Too many concurrent recoverable errors", NULL, NULL);
-}
-
-static struct mce_info *mce_find_info(void)
-{
- struct mce_info *mi;
-
- for (mi = mce_info; mi < &mce_info[MCE_INFO_MAX]; mi++)
- if (atomic_read(&mi->inuse) && mi->t == current)
- return mi;
- return NULL;
-}
-
-static void mce_clear_info(struct mce_info *mi)
-{
- atomic_set(&mi->inuse, 0);
-}
-
/*
* The actual machine check handler. This only handles real
* exceptions when something got corrupted coming in through int 18.
{
struct mca_config *cfg = &mca_cfg;
struct mce m, *final;
+ enum ctx_state prev_state;
int i;
int worst = 0;
int severity;
DECLARE_BITMAP(toclear, MAX_NR_BANKS);
DECLARE_BITMAP(valid_banks, MAX_NR_BANKS);
char *msg = "Unknown";
+ u64 recover_paddr = ~0ull;
+ int flags = MF_ACTION_REQUIRED;
+
+ prev_state = ist_enter(regs);
this_cpu_inc(mce_exception_count);
if (no_way_out)
mce_panic("Fatal machine check on current CPU", &m, msg);
if (worst == MCE_AR_SEVERITY) {
- /* schedule action before return to userland */
- mce_save_info(m.addr, m.mcgstatus & MCG_STATUS_RIPV);
- set_thread_flag(TIF_MCE_NOTIFY);
+ recover_paddr = m.addr;
+ if (!(m.mcgstatus & MCG_STATUS_RIPV))
+ flags |= MF_MUST_KILL;
} else if (kill_it) {
force_sig(SIGBUS, current);
}
mce_wrmsrl(MSR_IA32_MCG_STATUS, 0);
out:
sync_core();
+
+ if (recover_paddr == ~0ull)
+ goto done;
+
+ pr_err("Uncorrected hardware memory error in user-access at %llx",
+ recover_paddr);
+ /*
+ * We must call memory_failure() here even if the current process is
+ * doomed. We still need to mark the page as poisoned and alert any
+ * other users of the page.
+ */
+ ist_begin_non_atomic(regs);
+ local_irq_enable();
+ if (memory_failure(recover_paddr >> PAGE_SHIFT, MCE_VECTOR, flags) < 0) {
+ pr_err("Memory error not recovered");
+ force_sig(SIGBUS, current);
+ }
+ local_irq_disable();
+ ist_end_non_atomic();
+done:
+ ist_exit(regs, prev_state);
}
EXPORT_SYMBOL_GPL(do_machine_check);
}
#endif
-/*
- * Called in process context that interrupted by MCE and marked with
- * TIF_MCE_NOTIFY, just before returning to erroneous userland.
- * This code is allowed to sleep.
- * Attempt possible recovery such as calling the high level VM handler to
- * process any corrupted pages, and kill/signal current process if required.
- * Action required errors are handled here.
- */
-void mce_notify_process(void)
-{
- unsigned long pfn;
- struct mce_info *mi = mce_find_info();
- int flags = MF_ACTION_REQUIRED;
-
- if (!mi)
- mce_panic("Lost physical address for unconsumed uncorrectable error", NULL, NULL);
- pfn = mi->paddr >> PAGE_SHIFT;
-
- clear_thread_flag(TIF_MCE_NOTIFY);
-
- pr_err("Uncorrected hardware memory error in user-access at %llx",
- mi->paddr);
- /*
- * We must call memory_failure() here even if the current process is
- * doomed. We still need to mark the page as poisoned and alert any
- * other users of the page.
- */
- if (!mi->restartable)
- flags |= MF_MUST_KILL;
- if (memory_failure(pfn, MCE_VECTOR, flags) < 0) {
- pr_err("Memory error not recovered");
- force_sig(SIGBUS, current);
- }
- mce_clear_info(mi);
-}
-
/*
* Action optional processing happens here (picking up
* from the list of faulting pages that do_machine_check()
#include <linux/smp.h>
#include <asm/processor.h>
+#include <asm/traps.h>
#include <asm/mce.h>
#include <asm/msr.h>
/* Machine check handler for Pentium class Intel CPUs: */
static void pentium_machine_check(struct pt_regs *regs, long error_code)
{
+ enum ctx_state prev_state;
u32 loaddr, hi, lotype;
+ prev_state = ist_enter(regs);
+
rdmsr(MSR_IA32_P5_MC_ADDR, loaddr, hi);
rdmsr(MSR_IA32_P5_MC_TYPE, lotype, hi);
}
add_taint(TAINT_MACHINE_CHECK, LOCKDEP_NOW_UNRELIABLE);
+
+ ist_exit(regs, prev_state);
}
/* Set up machine check reporting for processors with Intel style MCE: */
#include <linux/types.h>
#include <asm/processor.h>
+#include <asm/traps.h>
#include <asm/mce.h>
#include <asm/msr.h>
/* Machine check handler for WinChip C6: */
static void winchip_machine_check(struct pt_regs *regs, long error_code)
{
+ enum ctx_state prev_state = ist_enter(regs);
+
printk(KERN_EMERG "CPU0: Machine Check Exception.\n");
add_taint(TAINT_MACHINE_CHECK, LOCKDEP_NOW_UNRELIABLE);
+
+ ist_exit(regs, prev_state);
}
/* Set up machine check reporting on the Winchip C6 series */
int error;
if (paravirt_enabled() || dis_ucode_ldr)
- return 0;
+ return -EINVAL;
if (c->x86_vendor == X86_VENDOR_INTEL)
microcode_ops = init_intel_microcode();
break;
case 55: /* 22nm Atom "Silvermont" */
+ case 76: /* 14nm Atom "Airmont" */
case 77: /* 22nm Atom "Silvermont Avoton/Rangely" */
memcpy(hw_cache_event_ids, slm_hw_cache_event_ids,
sizeof(hw_cache_event_ids));
* or use ldexp(count, -32).
* Watts = Joules/Time delta
*/
- return v << (32 - __this_cpu_read(rapl_pmu->hw_unit));
+ return v << (32 - __this_cpu_read(rapl_pmu)->hw_unit);
}
static u64 rapl_event_update(struct perf_event *event)
box->phys_id = phys_id;
box->pci_dev = pdev;
box->pmu = pmu;
- uncore_box_init(box);
pci_set_drvdata(pdev, box);
raw_spin_lock(&uncore_box_lock);
pmu = &type->pmus[j];
box = *per_cpu_ptr(pmu->box, cpu);
/* called by uncore_cpu_init? */
- if (box && box->phys_id >= 0) {
- uncore_box_init(box);
+ if (box && box->phys_id >= 0)
continue;
- }
for_each_online_cpu(k) {
exist = *per_cpu_ptr(pmu->box, k);
}
}
- if (box) {
+ if (box)
box->phys_id = phys_id;
- uncore_box_init(box);
- }
}
}
return 0;
return box->pmu->type->num_counters;
}
+static inline void uncore_box_init(struct intel_uncore_box *box)
+{
+ if (!test_and_set_bit(UNCORE_BOX_FLAG_INITIATED, &box->flags)) {
+ if (box->pmu->type->ops->init_box)
+ box->pmu->type->ops->init_box(box);
+ }
+}
+
static inline void uncore_disable_box(struct intel_uncore_box *box)
{
if (box->pmu->type->ops->disable_box)
static inline void uncore_enable_box(struct intel_uncore_box *box)
{
+ uncore_box_init(box);
+
if (box->pmu->type->ops->enable_box)
box->pmu->type->ops->enable_box(box);
}
return box->pmu->type->ops->read_counter(box, event);
}
-static inline void uncore_box_init(struct intel_uncore_box *box)
-{
- if (!test_and_set_bit(UNCORE_BOX_FLAG_INITIATED, &box->flags)) {
- if (box->pmu->type->ops->init_box)
- box->pmu->type->ops->init_box(box);
- }
-}
-
static inline bool uncore_box_is_fake(struct intel_uncore_box *box)
{
return (box->phys_id < 0);
movq \tmp,RSP+\offset(%rsp)
movq $__USER_DS,SS+\offset(%rsp)
movq $__USER_CS,CS+\offset(%rsp)
- movq $-1,RCX+\offset(%rsp)
+ movq RIP+\offset(%rsp),\tmp /* get rip */
+ movq \tmp,RCX+\offset(%rsp) /* copy it to rcx as sysret would do */
movq R11+\offset(%rsp),\tmp /* get eflags */
movq \tmp,EFLAGS+\offset(%rsp)
.endm
movq \tmp,R11+\offset(%rsp)
.endm
- .macro FAKE_STACK_FRAME child_rip
- /* push in order ss, rsp, eflags, cs, rip */
- xorl %eax, %eax
- pushq_cfi $__KERNEL_DS /* ss */
- /*CFI_REL_OFFSET ss,0*/
- pushq_cfi %rax /* rsp */
- CFI_REL_OFFSET rsp,0
- pushq_cfi $(X86_EFLAGS_IF|X86_EFLAGS_FIXED) /* eflags - interrupts on */
- /*CFI_REL_OFFSET rflags,0*/
- pushq_cfi $__KERNEL_CS /* cs */
- /*CFI_REL_OFFSET cs,0*/
- pushq_cfi \child_rip /* rip */
- CFI_REL_OFFSET rip,0
- pushq_cfi %rax /* orig rax */
- .endm
-
- .macro UNFAKE_STACK_FRAME
- addq $8*6, %rsp
- CFI_ADJUST_CFA_OFFSET -(6*8)
- .endm
-
/*
* initial frame state for interrupts (and exceptions without error code)
*/
CFI_REL_OFFSET r15, R15+\offset
.endm
-/* save partial stack frame */
- .macro SAVE_ARGS_IRQ
- cld
- /* start from rbp in pt_regs and jump over */
- movq_cfi rdi, (RDI-RBP)
- movq_cfi rsi, (RSI-RBP)
- movq_cfi rdx, (RDX-RBP)
- movq_cfi rcx, (RCX-RBP)
- movq_cfi rax, (RAX-RBP)
- movq_cfi r8, (R8-RBP)
- movq_cfi r9, (R9-RBP)
- movq_cfi r10, (R10-RBP)
- movq_cfi r11, (R11-RBP)
-
- /* Save rbp so that we can unwind from get_irq_regs() */
- movq_cfi rbp, 0
-
- /* Save previous stack value */
- movq %rsp, %rsi
-
- leaq -RBP(%rsp),%rdi /* arg1 for handler */
- testl $3, CS-RBP(%rsi)
- je 1f
- SWAPGS
- /*
- * irq_count is used to check if a CPU is already on an interrupt stack
- * or not. While this is essentially redundant with preempt_count it is
- * a little cheaper to use a separate counter in the PDA (short of
- * moving irq_enter into assembly, which would be too much work)
- */
-1: incl PER_CPU_VAR(irq_count)
- cmovzq PER_CPU_VAR(irq_stack_ptr),%rsp
- CFI_DEF_CFA_REGISTER rsi
-
- /* Store previous stack value */
- pushq %rsi
- CFI_ESCAPE 0x0f /* DW_CFA_def_cfa_expression */, 6, \
- 0x77 /* DW_OP_breg7 */, 0, \
- 0x06 /* DW_OP_deref */, \
- 0x08 /* DW_OP_const1u */, SS+8-RBP, \
- 0x22 /* DW_OP_plus */
- /* We entered an interrupt context - irqs are off: */
- TRACE_IRQS_OFF
- .endm
-
ENTRY(save_paranoid)
XCPT_FRAME 1 RDI+8
cld
* Has incomplete stack frame and undefined top of stack.
*/
ret_from_sys_call:
- movl $_TIF_ALLWORK_MASK,%edi
- /* edi: flagmask */
-sysret_check:
+ testl $_TIF_ALLWORK_MASK,TI_flags+THREAD_INFO(%rsp,RIP-ARGOFFSET)
+ jnz int_ret_from_sys_call_fixup /* Go the the slow path */
+
LOCKDEP_SYS_EXIT
DISABLE_INTERRUPTS(CLBR_NONE)
TRACE_IRQS_OFF
- movl TI_flags+THREAD_INFO(%rsp,RIP-ARGOFFSET),%edx
- andl %edi,%edx
- jnz sysret_careful
CFI_REMEMBER_STATE
/*
* sysretq will re-enable interrupts:
USERGS_SYSRET64
CFI_RESTORE_STATE
- /* Handle reschedules */
- /* edx: work, edi: workmask */
-sysret_careful:
- bt $TIF_NEED_RESCHED,%edx
- jnc sysret_signal
- TRACE_IRQS_ON
- ENABLE_INTERRUPTS(CLBR_NONE)
- pushq_cfi %rdi
- SCHEDULE_USER
- popq_cfi %rdi
- jmp sysret_check
- /* Handle a signal */
-sysret_signal:
- TRACE_IRQS_ON
- ENABLE_INTERRUPTS(CLBR_NONE)
-#ifdef CONFIG_AUDITSYSCALL
- bt $TIF_SYSCALL_AUDIT,%edx
- jc sysret_audit
-#endif
- /*
- * We have a signal, or exit tracing or single-step.
- * These all wind up with the iret return path anyway,
- * so just join that path right now.
- */
+int_ret_from_sys_call_fixup:
FIXUP_TOP_OF_STACK %r11, -ARGOFFSET
- jmp int_check_syscall_exit_work
-
-#ifdef CONFIG_AUDITSYSCALL
- /*
- * Return fast path for syscall audit. Call __audit_syscall_exit()
- * directly and then jump back to the fast path with TIF_SYSCALL_AUDIT
- * masked off.
- */
-sysret_audit:
- movq RAX-ARGOFFSET(%rsp),%rsi /* second arg, syscall return value */
- cmpq $-MAX_ERRNO,%rsi /* is it < -MAX_ERRNO? */
- setbe %al /* 1 if so, 0 if not */
- movzbl %al,%edi /* zero-extend that into %edi */
- call __audit_syscall_exit
- movl $(_TIF_ALLWORK_MASK & ~_TIF_SYSCALL_AUDIT),%edi
- jmp sysret_check
-#endif /* CONFIG_AUDITSYSCALL */
+ jmp int_ret_from_sys_call
/* Do syscall tracing */
tracesys:
FORK_LIKE vfork
FIXED_FRAME stub_iopl, sys_iopl
-ENTRY(ptregscall_common)
- DEFAULT_FRAME 1 8 /* offset 8: return address */
- RESTORE_TOP_OF_STACK %r11, 8
- movq_cfi_restore R15+8, r15
- movq_cfi_restore R14+8, r14
- movq_cfi_restore R13+8, r13
- movq_cfi_restore R12+8, r12
- movq_cfi_restore RBP+8, rbp
- movq_cfi_restore RBX+8, rbx
- ret $REST_SKIP /* pop extended registers */
- CFI_ENDPROC
-END(ptregscall_common)
-
ENTRY(stub_execve)
CFI_STARTPROC
addq $8, %rsp
/* reserve pt_regs for scratch regs and rbp */
subq $ORIG_RAX-RBP, %rsp
CFI_ADJUST_CFA_OFFSET ORIG_RAX-RBP
- SAVE_ARGS_IRQ
+ cld
+ /* start from rbp in pt_regs and jump over */
+ movq_cfi rdi, (RDI-RBP)
+ movq_cfi rsi, (RSI-RBP)
+ movq_cfi rdx, (RDX-RBP)
+ movq_cfi rcx, (RCX-RBP)
+ movq_cfi rax, (RAX-RBP)
+ movq_cfi r8, (R8-RBP)
+ movq_cfi r9, (R9-RBP)
+ movq_cfi r10, (R10-RBP)
+ movq_cfi r11, (R11-RBP)
+
+ /* Save rbp so that we can unwind from get_irq_regs() */
+ movq_cfi rbp, 0
+
+ /* Save previous stack value */
+ movq %rsp, %rsi
+
+ leaq -RBP(%rsp),%rdi /* arg1 for handler */
+ testl $3, CS-RBP(%rsi)
+ je 1f
+ SWAPGS
+ /*
+ * irq_count is used to check if a CPU is already on an interrupt stack
+ * or not. While this is essentially redundant with preempt_count it is
+ * a little cheaper to use a separate counter in the PDA (short of
+ * moving irq_enter into assembly, which would be too much work)
+ */
+1: incl PER_CPU_VAR(irq_count)
+ cmovzq PER_CPU_VAR(irq_stack_ptr),%rsp
+ CFI_DEF_CFA_REGISTER rsi
+
+ /* Store previous stack value */
+ pushq %rsi
+ CFI_ESCAPE 0x0f /* DW_CFA_def_cfa_expression */, 6, \
+ 0x77 /* DW_OP_breg7 */, 0, \
+ 0x06 /* DW_OP_deref */, \
+ 0x08 /* DW_OP_const1u */, SS+8-RBP, \
+ 0x22 /* DW_OP_plus */
+ /* We entered an interrupt context - irqs are off: */
+ TRACE_IRQS_OFF
+
call \func
.endm
*/
DISABLE_INTERRUPTS(CLBR_ANY)
TRACE_IRQS_IRETQ
+
+ /*
+ * Try to use SYSRET instead of IRET if we're returning to
+ * a completely clean 64-bit userspace context.
+ */
+ movq (RCX-R11)(%rsp), %rcx
+ cmpq %rcx,(RIP-R11)(%rsp) /* RCX == RIP */
+ jne opportunistic_sysret_failed
+
+ /*
+ * On Intel CPUs, sysret with non-canonical RCX/RIP will #GP
+ * in kernel space. This essentially lets the user take over
+ * the kernel, since userspace controls RSP. It's not worth
+ * testing for canonicalness exactly -- this check detects any
+ * of the 17 high bits set, which is true for non-canonical
+ * or kernel addresses. (This will pessimize vsyscall=native.
+ * Big deal.)
+ *
+ * If virtual addresses ever become wider, this will need
+ * to be updated to remain correct on both old and new CPUs.
+ */
+ .ifne __VIRTUAL_MASK_SHIFT - 47
+ .error "virtual address width changed -- sysret checks need update"
+ .endif
+ shr $__VIRTUAL_MASK_SHIFT, %rcx
+ jnz opportunistic_sysret_failed
+
+ cmpq $__USER_CS,(CS-R11)(%rsp) /* CS must match SYSRET */
+ jne opportunistic_sysret_failed
+
+ movq (R11-ARGOFFSET)(%rsp), %r11
+ cmpq %r11,(EFLAGS-ARGOFFSET)(%rsp) /* R11 == RFLAGS */
+ jne opportunistic_sysret_failed
+
+ testq $X86_EFLAGS_RF,%r11 /* sysret can't restore RF */
+ jnz opportunistic_sysret_failed
+
+ /* nothing to check for RSP */
+
+ cmpq $__USER_DS,(SS-ARGOFFSET)(%rsp) /* SS must match SYSRET */
+ jne opportunistic_sysret_failed
+
+ /*
+ * We win! This label is here just for ease of understanding
+ * perf profiles. Nothing jumps here.
+ */
+irq_return_via_sysret:
+ CFI_REMEMBER_STATE
+ RESTORE_ARGS 1,8,1
+ movq (RSP-RIP)(%rsp),%rsp
+ USERGS_SYSRET64
+ CFI_RESTORE_STATE
+
+opportunistic_sysret_failed:
SWAPGS
jmp restore_args
CFI_ADJUST_CFA_OFFSET ORIG_RAX-R15
.if \paranoid
+ .if \paranoid == 1
+ CFI_REMEMBER_STATE
+ testl $3, CS(%rsp) /* If coming from userspace, switch */
+ jnz 1f /* stacks. */
+ .endif
call save_paranoid
.else
call error_entry
jmp error_exit /* %ebx: no swapgs flag */
.endif
+ .if \paranoid == 1
+ CFI_RESTORE_STATE
+ /*
+ * Paranoid entry from userspace. Switch stacks and treat it
+ * as a normal entry. This means that paranoid handlers
+ * run in real process context if user_mode(regs).
+ */
+1:
+ call error_entry
+
+ DEFAULT_FRAME 0
+
+ movq %rsp,%rdi /* pt_regs pointer */
+ call sync_regs
+ movq %rax,%rsp /* switch stack */
+
+ movq %rsp,%rdi /* pt_regs pointer */
+
+ .if \has_error_code
+ movq ORIG_RAX(%rsp),%rsi /* get error code */
+ movq $-1,ORIG_RAX(%rsp) /* no syscall to restart */
+ .else
+ xorl %esi,%esi /* no error code */
+ .endif
+
+ call \do_sym
+
+ jmp error_exit /* %ebx: no swapgs flag */
+ .endif
+
CFI_ENDPROC
END(\sym)
.endm
idtentry bounds do_bounds has_error_code=0
idtentry invalid_op do_invalid_op has_error_code=0
idtentry device_not_available do_device_not_available has_error_code=0
-idtentry double_fault do_double_fault has_error_code=1 paranoid=1
+idtentry double_fault do_double_fault has_error_code=1 paranoid=2
idtentry coprocessor_segment_overrun do_coprocessor_segment_overrun has_error_code=0
idtentry invalid_TSS do_invalid_TSS has_error_code=1
idtentry segment_not_present do_segment_not_present has_error_code=1
#endif
/*
- * "Paranoid" exit path from exception stack.
- * Paranoid because this is used by NMIs and cannot take
- * any kernel state for granted.
- * We don't do kernel preemption checks here, because only
- * NMI should be common and it does not enable IRQs and
- * cannot get reschedule ticks.
+ * "Paranoid" exit path from exception stack. This is invoked
+ * only on return from non-NMI IST interrupts that came
+ * from kernel space.
*
- * "trace" is 0 for the NMI handler only, because irq-tracing
- * is fundamentally NMI-unsafe. (we cannot change the soft and
- * hard flags at once, atomically)
+ * We may be returning to very strange contexts (e.g. very early
+ * in syscall entry), so checking for preemption here would
+ * be complicated. Fortunately, we there's no good reason
+ * to try to handle preemption here.
*/
/* ebx: no swapgs flag */
TRACE_IRQS_OFF_DEBUG
testl %ebx,%ebx /* swapgs needed? */
jnz paranoid_restore
- testl $3,CS(%rsp)
- jnz paranoid_userspace
-paranoid_swapgs:
TRACE_IRQS_IRETQ 0
SWAPGS_UNSAFE_STACK
RESTORE_ALL 8
- jmp irq_return
+ INTERRUPT_RETURN
paranoid_restore:
TRACE_IRQS_IRETQ_DEBUG 0
RESTORE_ALL 8
- jmp irq_return
-paranoid_userspace:
- GET_THREAD_INFO(%rcx)
- movl TI_flags(%rcx),%ebx
- andl $_TIF_WORK_MASK,%ebx
- jz paranoid_swapgs
- movq %rsp,%rdi /* &pt_regs */
- call sync_regs
- movq %rax,%rsp /* switch stack for scheduling */
- testl $_TIF_NEED_RESCHED,%ebx
- jnz paranoid_schedule
- movl %ebx,%edx /* arg3: thread flags */
- TRACE_IRQS_ON
- ENABLE_INTERRUPTS(CLBR_NONE)
- xorl %esi,%esi /* arg2: oldset */
- movq %rsp,%rdi /* arg1: &pt_regs */
- call do_notify_resume
- DISABLE_INTERRUPTS(CLBR_NONE)
- TRACE_IRQS_OFF
- jmp paranoid_userspace
-paranoid_schedule:
- TRACE_IRQS_ON
- ENABLE_INTERRUPTS(CLBR_ANY)
- SCHEDULE_USER
- DISABLE_INTERRUPTS(CLBR_ANY)
- TRACE_IRQS_OFF
- jmp paranoid_userspace
+ INTERRUPT_RETURN
CFI_ENDPROC
END(paranoid_exit)
*dr7 |= encode_dr7(i, info->len, info->type);
set_debugreg(*dr7, 7);
+ if (info->mask)
+ set_dr_addr_mask(info->mask, i);
return 0;
}
*dr7 &= ~__encode_dr7(i, info->len, info->type);
set_debugreg(*dr7, 7);
-}
-
-static int get_hbp_len(u8 hbp_len)
-{
- unsigned int len_in_bytes = 0;
-
- switch (hbp_len) {
- case X86_BREAKPOINT_LEN_1:
- len_in_bytes = 1;
- break;
- case X86_BREAKPOINT_LEN_2:
- len_in_bytes = 2;
- break;
- case X86_BREAKPOINT_LEN_4:
- len_in_bytes = 4;
- break;
-#ifdef CONFIG_X86_64
- case X86_BREAKPOINT_LEN_8:
- len_in_bytes = 8;
- break;
-#endif
- }
- return len_in_bytes;
+ if (info->mask)
+ set_dr_addr_mask(0, i);
}
/*
struct arch_hw_breakpoint *info = counter_arch_bp(bp);
va = info->address;
- len = get_hbp_len(info->len);
+ len = bp->attr.bp_len;
return (va >= TASK_SIZE) && ((va + len - 1) >= TASK_SIZE);
}
}
/* Len */
+ info->mask = 0;
+
switch (bp->attr.bp_len) {
case HW_BREAKPOINT_LEN_1:
info->len = X86_BREAKPOINT_LEN_1;
break;
#endif
default:
- return -EINVAL;
+ if (!is_power_of_2(bp->attr.bp_len))
+ return -EINVAL;
+ if (!cpu_has_bpext)
+ return -EOPNOTSUPP;
+ info->mask = bp->attr.bp_len - 1;
+ info->len = X86_BREAKPOINT_LEN_1;
}
return 0;
}
+
/*
* Validate the arch-specific HW Breakpoint register settings
*/
if (ret)
return ret;
- ret = -EINVAL;
-
switch (info->len) {
case X86_BREAKPOINT_LEN_1:
align = 0;
+ if (info->mask)
+ align = info->mask;
break;
case X86_BREAKPOINT_LEN_2:
align = 1;
break;
#endif
default:
- return ret;
+ WARN_ON_ONCE(1);
}
/*
: "memory", "cc", "edx", "ecx", "eax");
}
-/* how to get the current stack pointer from C */
-#define current_stack_pointer ({ \
- unsigned long sp; \
- asm("mov %%esp,%0" : "=g" (sp)); \
- sp; \
-})
-
static inline void *current_stack(void)
{
- return (void *)(current_stack_pointer & ~(THREAD_SIZE - 1));
+ return (void *)(current_stack_pointer() & ~(THREAD_SIZE - 1));
}
static inline int
/* Save the next esp at the bottom of the stack */
prev_esp = (u32 *)irqstk;
- *prev_esp = current_stack_pointer;
+ *prev_esp = current_stack_pointer();
if (unlikely(overflow))
call_on_stack(print_stack_overflow, isp);
/* Push the previous esp onto the stack */
prev_esp = (u32 *)irqstk;
- *prev_esp = current_stack_pointer;
+ *prev_esp = current_stack_pointer();
call_on_stack(__do_softirq, isp);
}
static __init int add_rtc_cmos(void)
{
#ifdef CONFIG_PNP
- static const char * const const ids[] __initconst =
+ static const char * const ids[] __initconst =
{ "PNP0b00", "PNP0b01", "PNP0b02", };
struct pnp_dev *dev;
struct pnp_id *id;
{
user_exit();
-#ifdef CONFIG_X86_MCE
- /* notify userspace of pending MCEs */
- if (thread_info_flags & _TIF_MCE_NOTIFY)
- mce_notify_process();
-#endif /* CONFIG_X86_64 && CONFIG_X86_MCE */
-
if (thread_info_flags & _TIF_UPROBE)
uprobe_notify_resume(regs);
#include <asm/setup.h>
#include <asm/uv/uv.h>
#include <linux/mc146818rtc.h>
-#include <asm/smpboot_hooks.h>
#include <asm/i8259.h>
#include <asm/realmode.h>
#include <asm/misc.h>
atomic_t init_deasserted;
+static inline void smpboot_setup_warm_reset_vector(unsigned long start_eip)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&rtc_lock, flags);
+ CMOS_WRITE(0xa, 0xf);
+ spin_unlock_irqrestore(&rtc_lock, flags);
+ local_flush_tlb();
+ pr_debug("1.\n");
+ *((volatile unsigned short *)phys_to_virt(TRAMPOLINE_PHYS_HIGH)) =
+ start_eip >> 4;
+ pr_debug("2.\n");
+ *((volatile unsigned short *)phys_to_virt(TRAMPOLINE_PHYS_LOW)) =
+ start_eip & 0xf;
+ pr_debug("3.\n");
+}
+
+static inline void smpboot_restore_warm_reset_vector(void)
+{
+ unsigned long flags;
+
+ /*
+ * Install writable page 0 entry to set BIOS data area.
+ */
+ local_flush_tlb();
+
+ /*
+ * Paranoid: Set warm reset code and vector here back
+ * to default values.
+ */
+ spin_lock_irqsave(&rtc_lock, flags);
+ CMOS_WRITE(0, 0xf);
+ spin_unlock_irqrestore(&rtc_lock, flags);
+
+ *((volatile u32 *)phys_to_virt(TRAMPOLINE_PHYS_LOW)) = 0;
+}
+
/*
* Report back to the Boot Processor during boot time or to the caller processor
* during CPU online.
* CPU, first the APIC. (this is probably redundant on most
* boards)
*/
- setup_local_APIC();
- end_local_APIC_setup();
+ apic_ap_setup();
/*
* Need to setup vector mappings before we enable interrupts.
*/
static __init void disable_smp(void)
{
+ pr_info("SMP disabled\n");
+
+ disable_ioapic_support();
+
init_cpu_present(cpumask_of(0));
init_cpu_possible(cpumask_of(0));
- smpboot_clear_io_apic_irqs();
if (smp_found_config)
physid_set_mask_of_physid(boot_cpu_physical_apicid, &phys_cpu_present_map);
cpumask_set_cpu(0, cpu_core_mask(0));
}
+enum {
+ SMP_OK,
+ SMP_NO_CONFIG,
+ SMP_NO_APIC,
+ SMP_FORCE_UP,
+};
+
/*
* Various sanity checks.
*/
if (!smp_found_config && !acpi_lapic) {
preempt_enable();
pr_notice("SMP motherboard not detected\n");
- disable_smp();
- if (APIC_init_uniprocessor())
- pr_notice("Local APIC not detected. Using dummy APIC emulation.\n");
- return -1;
+ return SMP_NO_CONFIG;
}
/*
boot_cpu_physical_apicid);
pr_err("... forcing use of dummy APIC emulation (tell your hw vendor)\n");
}
- smpboot_clear_io_apic();
- disable_ioapic_support();
- return -1;
+ return SMP_NO_APIC;
}
verify_local_APIC();
*/
if (!max_cpus) {
pr_info("SMP mode deactivated\n");
- smpboot_clear_io_apic();
-
- connect_bsp_APIC();
- setup_local_APIC();
- bsp_end_local_APIC_setup();
- return -1;
+ return SMP_FORCE_UP;
}
- return 0;
+ return SMP_OK;
}
static void __init smp_cpu_index_default(void)
}
set_cpu_sibling_map(0);
- if (smp_sanity_check(max_cpus) < 0) {
- pr_info("SMP disabled\n");
+ switch (smp_sanity_check(max_cpus)) {
+ case SMP_NO_CONFIG:
disable_smp();
+ if (APIC_init_uniprocessor())
+ pr_notice("Local APIC not detected. Using dummy APIC emulation.\n");
return;
+ case SMP_NO_APIC:
+ disable_smp();
+ return;
+ case SMP_FORCE_UP:
+ disable_smp();
+ apic_bsp_setup(false);
+ return;
+ case SMP_OK:
+ break;
}
default_setup_apic_routing();
/* Or can we switch back to PIC here? */
}
- connect_bsp_APIC();
-
- /*
- * Switch from PIC to APIC mode.
- */
- setup_local_APIC();
-
- if (x2apic_mode)
- cpu0_logical_apicid = apic_read(APIC_LDR);
- else
- cpu0_logical_apicid = GET_APIC_LOGICAL_ID(apic_read(APIC_LDR));
-
- /*
- * Enable IO APIC before setting up error vector
- */
- if (!skip_ioapic_setup && nr_ioapics)
- enable_IO_APIC();
-
- bsp_end_local_APIC_setup();
- smpboot_setup_io_apic();
- /*
- * Set up local APIC timer on boot CPU.
- */
+ cpu0_logical_apicid = apic_bsp_setup(false);
pr_info("CPU%d: ", 0);
print_cpu_info(&cpu_data(0));
- x86_init.timers.setup_percpu_clockev();
if (is_uv_system())
uv_system_init();
nmi_selftest();
impress_friends();
-#ifdef CONFIG_X86_IO_APIC
setup_ioapic_dest();
-#endif
mtrr_aps_init();
}
preempt_count_dec();
}
+enum ctx_state ist_enter(struct pt_regs *regs)
+{
+ enum ctx_state prev_state;
+
+ if (user_mode_vm(regs)) {
+ /* Other than that, we're just an exception. */
+ prev_state = exception_enter();
+ } else {
+ /*
+ * We might have interrupted pretty much anything. In
+ * fact, if we're a machine check, we can even interrupt
+ * NMI processing. We don't want in_nmi() to return true,
+ * but we need to notify RCU.
+ */
+ rcu_nmi_enter();
+ prev_state = IN_KERNEL; /* the value is irrelevant. */
+ }
+
+ /*
+ * We are atomic because we're on the IST stack (or we're on x86_32,
+ * in which case we still shouldn't schedule).
+ *
+ * This must be after exception_enter(), because exception_enter()
+ * won't do anything if in_interrupt() returns true.
+ */
+ preempt_count_add(HARDIRQ_OFFSET);
+
+ /* This code is a bit fragile. Test it. */
+ rcu_lockdep_assert(rcu_is_watching(), "ist_enter didn't work");
+
+ return prev_state;
+}
+
+void ist_exit(struct pt_regs *regs, enum ctx_state prev_state)
+{
+ /* Must be before exception_exit. */
+ preempt_count_sub(HARDIRQ_OFFSET);
+
+ if (user_mode_vm(regs))
+ return exception_exit(prev_state);
+ else
+ rcu_nmi_exit();
+}
+
+/**
+ * ist_begin_non_atomic() - begin a non-atomic section in an IST exception
+ * @regs: regs passed to the IST exception handler
+ *
+ * IST exception handlers normally cannot schedule. As a special
+ * exception, if the exception interrupted userspace code (i.e.
+ * user_mode_vm(regs) would return true) and the exception was not
+ * a double fault, it can be safe to schedule. ist_begin_non_atomic()
+ * begins a non-atomic section within an ist_enter()/ist_exit() region.
+ * Callers are responsible for enabling interrupts themselves inside
+ * the non-atomic section, and callers must call is_end_non_atomic()
+ * before ist_exit().
+ */
+void ist_begin_non_atomic(struct pt_regs *regs)
+{
+ BUG_ON(!user_mode_vm(regs));
+
+ /*
+ * Sanity check: we need to be on the normal thread stack. This
+ * will catch asm bugs and any attempt to use ist_preempt_enable
+ * from double_fault.
+ */
+ BUG_ON(((current_stack_pointer() ^ this_cpu_read_stable(kernel_stack))
+ & ~(THREAD_SIZE - 1)) != 0);
+
+ preempt_count_sub(HARDIRQ_OFFSET);
+}
+
+/**
+ * ist_end_non_atomic() - begin a non-atomic section in an IST exception
+ *
+ * Ends a non-atomic section started with ist_begin_non_atomic().
+ */
+void ist_end_non_atomic(void)
+{
+ preempt_count_add(HARDIRQ_OFFSET);
+}
+
static nokprobe_inline int
do_trap_no_signal(struct task_struct *tsk, int trapnr, char *str,
struct pt_regs *regs, long error_code)
* end up promoting it to a doublefault. In that case, modify
* the stack to make it look like we just entered the #GP
* handler from user space, similar to bad_iret.
+ *
+ * No need for ist_enter here because we don't use RCU.
*/
if (((long)regs->sp >> PGDIR_SHIFT) == ESPFIX_PGD_ENTRY &&
regs->cs == __KERNEL_CS &&
normal_regs->orig_ax = 0; /* Missing (lost) #GP error code */
regs->ip = (unsigned long)general_protection;
regs->sp = (unsigned long)&normal_regs->orig_ax;
+
return;
}
#endif
- exception_enter();
- /* Return not checked because double check cannot be ignored */
+ ist_enter(regs); /* Discard prev_state because we won't return. */
notify_die(DIE_TRAP, str, regs, error_code, X86_TRAP_DF, SIGSEGV);
tsk->thread.error_code = error_code;
if (poke_int3_handler(regs))
return;
- prev_state = exception_enter();
+ prev_state = ist_enter(regs);
#ifdef CONFIG_KGDB_LOW_LEVEL_TRAP
if (kgdb_ll_trap(DIE_INT3, "int3", regs, error_code, X86_TRAP_BP,
SIGTRAP) == NOTIFY_STOP)
preempt_conditional_cli(regs);
debug_stack_usage_dec();
exit:
- exception_exit(prev_state);
+ ist_exit(regs, prev_state);
}
NOKPROBE_SYMBOL(do_int3);
#ifdef CONFIG_X86_64
/*
- * Help handler running on IST stack to switch back to user stack
- * for scheduling or signal handling. The actual stack switch is done in
- * entry.S
+ * Help handler running on IST stack to switch off the IST stack if the
+ * interrupted code was in user mode. The actual stack switch is done in
+ * entry_64.S
*/
asmlinkage __visible notrace struct pt_regs *sync_regs(struct pt_regs *eregs)
{
- struct pt_regs *regs = eregs;
- /* Did already sync */
- if (eregs == (struct pt_regs *)eregs->sp)
- ;
- /* Exception from user space */
- else if (user_mode(eregs))
- regs = task_pt_regs(current);
- /*
- * Exception from kernel and interrupts are enabled. Move to
- * kernel process stack.
- */
- else if (eregs->flags & X86_EFLAGS_IF)
- regs = (struct pt_regs *)(eregs->sp -= sizeof(struct pt_regs));
- if (eregs != regs)
- *regs = *eregs;
+ struct pt_regs *regs = task_pt_regs(current);
+ *regs = *eregs;
return regs;
}
NOKPROBE_SYMBOL(sync_regs);
unsigned long dr6;
int si_code;
- prev_state = exception_enter();
+ prev_state = ist_enter(regs);
get_debugreg(dr6, 6);
debug_stack_usage_dec();
exit:
- exception_exit(prev_state);
+ ist_exit(regs, prev_state);
}
NOKPROBE_SYMBOL(do_debug);
select HAVE_KVM_MSI
select HAVE_KVM_CPU_RELAX_INTERCEPT
select KVM_VFIO
+ select SRCU
---help---
Support hosting fully virtualized guest machines using hardware
virtualization extensions. You will need a fairly recent
u16 cid, lid;
u32 ldr, aid;
+ if (!kvm_apic_present(vcpu))
+ continue;
+
aid = kvm_apic_id(apic);
ldr = kvm_apic_get_reg(apic, APIC_LDR);
cid = apic_cluster_id(new, ldr);
if (fault & (VM_FAULT_SIGBUS|VM_FAULT_HWPOISON|
VM_FAULT_HWPOISON_LARGE))
do_sigbus(regs, error_code, address, fault);
+ else if (fault & VM_FAULT_SIGSEGV)
+ bad_area_nosemaphore(regs, error_code, address);
else
BUG();
}
DMI_MATCH(DMI_PRODUCT_NAME, "ftServer"),
},
},
+ {
+ .callback = set_scan_all,
+ .ident = "Stratus/NEC ftServer",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "NEC"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "Express5800/R32"),
+ },
+ },
+ {
+ .callback = set_scan_all,
+ .ident = "Stratus/NEC ftServer",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "NEC"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "Express5800/R31"),
+ },
+ },
{}
};
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x0801, mrst_power_off_unused_dev);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x0809, mrst_power_off_unused_dev);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x080C, mrst_power_off_unused_dev);
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x0812, mrst_power_off_unused_dev);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x0815, mrst_power_off_unused_dev);
/*
PHONY += vdso_install $(vdso_img_insttargets)
vdso_install: $(vdso_img_insttargets) FORCE
-clean-files := vdso32-syscall* vdso32-sysenter* vdso32-int80*
+clean-files := vdso32-syscall* vdso32-sysenter* vdso32-int80* vdso64*
if (unlikely(fault & VM_FAULT_ERROR)) {
if (fault & VM_FAULT_OOM)
goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
else if (fault & VM_FAULT_SIGBUS)
goto do_sigbus;
BUG();
static void blk_mq_sysfs_release(struct kobject *kobj)
{
- struct request_queue *q;
-
- q = container_of(kobj, struct request_queue, mq_kobj);
- free_percpu(q->queue_ctx);
-}
-
-static void blk_mq_ctx_release(struct kobject *kobj)
-{
- struct blk_mq_ctx *ctx;
-
- ctx = container_of(kobj, struct blk_mq_ctx, kobj);
- kobject_put(&ctx->queue->mq_kobj);
-}
-
-static void blk_mq_hctx_release(struct kobject *kobj)
-{
- struct blk_mq_hw_ctx *hctx;
-
- hctx = container_of(kobj, struct blk_mq_hw_ctx, kobj);
- kfree(hctx);
}
struct blk_mq_ctx_sysfs_entry {
static struct kobj_type blk_mq_ctx_ktype = {
.sysfs_ops = &blk_mq_sysfs_ops,
.default_attrs = default_ctx_attrs,
- .release = blk_mq_ctx_release,
+ .release = blk_mq_sysfs_release,
};
static struct kobj_type blk_mq_hw_ktype = {
.sysfs_ops = &blk_mq_hw_sysfs_ops,
.default_attrs = default_hw_ctx_attrs,
- .release = blk_mq_hctx_release,
+ .release = blk_mq_sysfs_release,
};
static void blk_mq_unregister_hctx(struct blk_mq_hw_ctx *hctx)
return ret;
hctx_for_each_ctx(hctx, ctx, i) {
- kobject_get(&q->mq_kobj);
ret = kobject_add(&ctx->kobj, &hctx->kobj, "cpu%u", ctx->cpu);
if (ret)
break;
mutex_unlock(&set->tag_list_lock);
}
+/*
+ * It is the actual release handler for mq, but we do it from
+ * request queue's release handler for avoiding use-after-free
+ * and headache because q->mq_kobj shouldn't have been introduced,
+ * but we can't group ctx/kctx kobj without it.
+ */
+void blk_mq_release(struct request_queue *q)
+{
+ struct blk_mq_hw_ctx *hctx;
+ unsigned int i;
+
+ /* hctx kobj stays in hctx */
+ queue_for_each_hw_ctx(q, hctx, i)
+ kfree(hctx);
+
+ kfree(q->queue_hw_ctx);
+
+ /* ctx kobj stays in queue_ctx */
+ free_percpu(q->queue_ctx);
+}
+
struct request_queue *blk_mq_init_queue(struct blk_mq_tag_set *set)
{
struct blk_mq_hw_ctx **hctxs;
percpu_ref_exit(&q->mq_usage_counter);
- kfree(q->queue_hw_ctx);
kfree(q->mq_map);
- q->queue_hw_ctx = NULL;
q->mq_map = NULL;
mutex_lock(&all_q_mutex);
extern void blk_mq_rq_timed_out(struct request *req, bool reserved);
+void blk_mq_release(struct request_queue *q);
+
/*
* Basic implementation of sparser bitmap, allowing the user to spread
* the bits over more cachelines.
if (!q->mq_ops)
blk_free_flush_queue(q->fq);
+ else
+ blk_mq_release(q);
blk_trace_shutdown(q);
source "drivers/platform/Kconfig"
-source "drivers/soc/Kconfig"
-
source "drivers/clk/Kconfig"
source "drivers/hwspinlock/Kconfig"
/*
* ACPI support for Intel Lynxpoint LPSS.
*
- * Copyright (C) 2013, 2014, Intel Corporation
+ * Copyright (C) 2013, Intel Corporation
* Authors: Mika Westerberg <mika.westerberg@linux.intel.com>
* Rafael J. Wysocki <rafael.j.wysocki@intel.com>
*
#define LPSS_CLK_DIVIDER BIT(2)
#define LPSS_LTR BIT(3)
#define LPSS_SAVE_CTX BIT(4)
-#define LPSS_DEV_PROXY BIT(5)
-#define LPSS_PROXY_REQ BIT(6)
struct lpss_private_data;
void (*setup)(struct lpss_private_data *pdata);
};
-static struct device *proxy_device;
-
static struct lpss_device_desc lpss_dma_desc = {
- .flags = LPSS_CLK | LPSS_PROXY_REQ,
+ .flags = LPSS_CLK,
};
struct lpss_private_data {
};
static struct lpss_device_desc byt_uart_dev_desc = {
- .flags = LPSS_CLK | LPSS_CLK_GATE | LPSS_CLK_DIVIDER | LPSS_SAVE_CTX |
- LPSS_DEV_PROXY,
+ .flags = LPSS_CLK | LPSS_CLK_GATE | LPSS_CLK_DIVIDER | LPSS_SAVE_CTX,
.prv_offset = 0x800,
.setup = lpss_uart_setup,
};
static struct lpss_device_desc byt_spi_dev_desc = {
- .flags = LPSS_CLK | LPSS_CLK_GATE | LPSS_CLK_DIVIDER | LPSS_SAVE_CTX |
- LPSS_DEV_PROXY,
+ .flags = LPSS_CLK | LPSS_CLK_GATE | LPSS_CLK_DIVIDER | LPSS_SAVE_CTX,
.prv_offset = 0x400,
};
static struct lpss_device_desc byt_sdio_dev_desc = {
- .flags = LPSS_CLK | LPSS_DEV_PROXY,
+ .flags = LPSS_CLK,
};
static struct lpss_device_desc byt_i2c_dev_desc = {
- .flags = LPSS_CLK | LPSS_SAVE_CTX | LPSS_DEV_PROXY,
+ .flags = LPSS_CLK | LPSS_SAVE_CTX,
.prv_offset = 0x800,
.setup = byt_i2c_setup,
};
adev->driver_data = pdata;
pdev = acpi_create_platform_device(adev);
if (!IS_ERR_OR_NULL(pdev)) {
- if (!proxy_device && dev_desc->flags & LPSS_DEV_PROXY)
- proxy_device = &pdev->dev;
return 1;
}
if (pdata->dev_desc->flags & LPSS_SAVE_CTX)
acpi_lpss_save_ctx(dev, pdata);
- ret = acpi_dev_runtime_suspend(dev);
- if (ret)
- return ret;
-
- if (pdata->dev_desc->flags & LPSS_PROXY_REQ && proxy_device)
- return pm_runtime_put_sync_suspend(proxy_device);
-
- return 0;
+ return acpi_dev_runtime_suspend(dev);
}
static int acpi_lpss_runtime_resume(struct device *dev)
struct lpss_private_data *pdata = acpi_driver_data(ACPI_COMPANION(dev));
int ret;
- if (pdata->dev_desc->flags & LPSS_PROXY_REQ && proxy_device) {
- ret = pm_runtime_get_sync(proxy_device);
- if (ret)
- return ret;
- }
-
ret = acpi_dev_runtime_resume(dev);
if (ret)
return ret;
struct regmap {
union {
struct mutex mutex;
- spinlock_t spinlock;
+ struct {
+ spinlock_t spinlock;
+ unsigned long spinlock_flags;
+ };
};
- unsigned long spinlock_flags;
regmap_lock lock;
regmap_unlock unlock;
void *lock_arg; /* This is passed to lock/unlock functions */
void regmap_async_complete_cb(struct regmap_async *async, int ret);
+enum regmap_endian regmap_get_val_endian(struct device *dev,
+ const struct regmap_bus *bus,
+ const struct regmap_config *config);
+
extern struct regcache_ops regcache_rbtree_ops;
extern struct regcache_ops regcache_lzo_ops;
extern struct regcache_ops regcache_flat_ops;
}
static const struct regmap_bus ac97_regmap_bus = {
- .reg_write = regmap_ac97_reg_write,
- .reg_read = regmap_ac97_reg_read,
+ .reg_write = regmap_ac97_reg_write,
+ .reg_read = regmap_ac97_reg_read,
};
/**
#include <linux/i2c.h>
#include <linux/module.h>
+#include "internal.h"
static int regmap_smbus_byte_reg_read(void *context, unsigned int reg,
unsigned int *val)
.reg_read = regmap_smbus_word_reg_read,
};
+static int regmap_smbus_word_read_swapped(void *context, unsigned int reg,
+ unsigned int *val)
+{
+ struct device *dev = context;
+ struct i2c_client *i2c = to_i2c_client(dev);
+ int ret;
+
+ if (reg > 0xff)
+ return -EINVAL;
+
+ ret = i2c_smbus_read_word_swapped(i2c, reg);
+ if (ret < 0)
+ return ret;
+
+ *val = ret;
+
+ return 0;
+}
+
+static int regmap_smbus_word_write_swapped(void *context, unsigned int reg,
+ unsigned int val)
+{
+ struct device *dev = context;
+ struct i2c_client *i2c = to_i2c_client(dev);
+
+ if (val > 0xffff || reg > 0xff)
+ return -EINVAL;
+
+ return i2c_smbus_write_word_swapped(i2c, reg, val);
+}
+
+static struct regmap_bus regmap_smbus_word_swapped = {
+ .reg_write = regmap_smbus_word_write_swapped,
+ .reg_read = regmap_smbus_word_read_swapped,
+};
+
static int regmap_i2c_write(void *context, const void *data, size_t count)
{
struct device *dev = context;
else if (config->val_bits == 16 && config->reg_bits == 8 &&
i2c_check_functionality(i2c->adapter,
I2C_FUNC_SMBUS_WORD_DATA))
- return ®map_smbus_word;
+ switch (regmap_get_val_endian(&i2c->dev, NULL, config)) {
+ case REGMAP_ENDIAN_LITTLE:
+ return ®map_smbus_word;
+ case REGMAP_ENDIAN_BIG:
+ return ®map_smbus_word_swapped;
+ default: /* everything else is not supported */
+ break;
+ }
else if (config->val_bits == 8 && config->reg_bits == 8 &&
i2c_check_functionality(i2c->adapter,
I2C_FUNC_SMBUS_BYTE_DATA))
return REGMAP_ENDIAN_BIG;
}
-static enum regmap_endian regmap_get_val_endian(struct device *dev,
- const struct regmap_bus *bus,
- const struct regmap_config *config)
+enum regmap_endian regmap_get_val_endian(struct device *dev,
+ const struct regmap_bus *bus,
+ const struct regmap_config *config)
{
struct device_node *np;
enum regmap_endian endian;
/* Use this if no other value was found */
return REGMAP_ENDIAN_BIG;
}
+EXPORT_SYMBOL_GPL(regmap_get_val_endian);
/**
* regmap_init(): Initialise register map
* If an image has a non-zero parent overlap, get a reference to its
* parent.
*
- * We must get the reference before checking for the overlap to
- * coordinate properly with zeroing the parent overlap in
- * rbd_dev_v2_parent_info() when an image gets flattened. We
- * drop it again if there is no overlap.
- *
* Returns true if the rbd device has a parent with a non-zero
* overlap and a reference for it was successfully taken, or
* false otherwise.
*/
static bool rbd_dev_parent_get(struct rbd_device *rbd_dev)
{
- int counter;
+ int counter = 0;
if (!rbd_dev->parent_spec)
return false;
- counter = atomic_inc_return_safe(&rbd_dev->parent_ref);
- if (counter > 0 && rbd_dev->parent_overlap)
- return true;
-
- /* Image was flattened, but parent is not yet torn down */
+ down_read(&rbd_dev->header_rwsem);
+ if (rbd_dev->parent_overlap)
+ counter = atomic_inc_return_safe(&rbd_dev->parent_ref);
+ up_read(&rbd_dev->header_rwsem);
if (counter < 0)
rbd_warn(rbd_dev, "parent reference overflow");
- return false;
+ return counter > 0;
}
/*
*/
if (rbd_dev->parent_overlap) {
rbd_dev->parent_overlap = 0;
- smp_mb();
rbd_dev_parent_put(rbd_dev);
pr_info("%s: clone image has been flattened\n",
rbd_dev->disk->disk_name);
* treat it specially.
*/
rbd_dev->parent_overlap = overlap;
- smp_mb();
if (!overlap) {
/* A null parent_spec indicates it's the initial probe */
{
struct rbd_image_header *header;
- /* Drop parent reference unless it's already been done (or none) */
-
- if (rbd_dev->parent_overlap)
- rbd_dev_parent_put(rbd_dev);
+ rbd_dev_parent_put(rbd_dev);
/* Free dynamic fields from the header, then zero it out */
__u32 c = f->pool[2], d = f->pool[3];
a += b; c += d;
- b = rol32(a, 6); d = rol32(c, 27);
+ b = rol32(b, 6); d = rol32(d, 27);
d ^= a; b ^= c;
a += b; c += d;
- b = rol32(a, 16); d = rol32(c, 14);
+ b = rol32(b, 16); d = rol32(d, 14);
d ^= a; b ^= c;
a += b; c += d;
- b = rol32(a, 6); d = rol32(c, 27);
+ b = rol32(b, 6); d = rol32(d, 27);
d ^= a; b ^= c;
a += b; c += d;
- b = rol32(a, 16); d = rol32(c, 14);
+ b = rol32(b, 16); d = rol32(d, 14);
d ^= a; b ^= c;
f->pool[0] = a; f->pool[1] = b;
bool
select HAVE_CLK_PREPARE
select CLKDEV_LOOKUP
+ select SRCU
---help---
The common clock framework is a single definition of struct
clk, useful across many platforms, as well as an
config CPU_FREQ
bool "CPU Frequency scaling"
+ select SRCU
help
CPU Frequency scaling allows you to change the clock speed of
CPUs on the fly. This is a nice method to save power, because
menuconfig PM_DEVFREQ
bool "Generic Dynamic Voltage and Frequency Scaling (DVFS) support"
+ select SRCU
help
A device may have a list of frequencies and voltages available.
devfreq, a generic DVFS framework can be registered for a device
client->irq = irq_of_parse_and_map(client->dev.of_node, 0);
} else {
pdata = dev_get_platdata(&client->dev);
- if (!pdata || !gpio_is_valid(pdata->base)) {
- dev_dbg(&client->dev, "invalid platform data\n");
- return -EINVAL;
+ if (!pdata) {
+ pdata = devm_kzalloc(&client->dev,
+ sizeof(struct mcp23s08_platform_data),
+ GFP_KERNEL);
+ pdata->base = -1;
}
}
} else {
type = spi_get_device_id(spi)->driver_data;
pdata = dev_get_platdata(&spi->dev);
- if (!pdata || !gpio_is_valid(pdata->base)) {
- dev_dbg(&spi->dev,
- "invalid or missing platform data\n");
- return -EINVAL;
+ if (!pdata) {
+ pdata = devm_kzalloc(&spi->dev,
+ sizeof(struct mcp23s08_platform_data),
+ GFP_KERNEL);
+ pdata->base = -1;
}
for (addr = 0; addr < ARRAY_SIZE(pdata->chip); addr++) {
#define BANK_USED(bank) (bank->mod_usage || bank->irq_usage)
#define LINE_USED(line, offset) (line & (BIT(offset)))
+static void omap_gpio_unmask_irq(struct irq_data *d);
+
static int omap_irq_to_gpio(struct gpio_bank *bank, unsigned int gpio_irq)
{
return bank->chip.base + gpio_irq;
return readl_relaxed(reg) & mask;
}
+static void omap_gpio_init_irq(struct gpio_bank *bank, unsigned gpio,
+ unsigned offset)
+{
+ if (!LINE_USED(bank->mod_usage, offset)) {
+ omap_enable_gpio_module(bank, offset);
+ omap_set_gpio_direction(bank, offset, 1);
+ }
+ bank->irq_usage |= BIT(GPIO_INDEX(bank, gpio));
+}
+
static int omap_gpio_irq_type(struct irq_data *d, unsigned type)
{
struct gpio_bank *bank = omap_irq_data_get_bank(d);
spin_lock_irqsave(&bank->lock, flags);
offset = GPIO_INDEX(bank, gpio);
retval = omap_set_gpio_triggering(bank, offset, type);
- if (!LINE_USED(bank->mod_usage, offset)) {
- omap_enable_gpio_module(bank, offset);
- omap_set_gpio_direction(bank, offset, 1);
- } else if (!omap_gpio_is_input(bank, BIT(offset))) {
+ omap_gpio_init_irq(bank, gpio, offset);
+ if (!omap_gpio_is_input(bank, BIT(offset))) {
spin_unlock_irqrestore(&bank->lock, flags);
return -EINVAL;
}
-
- bank->irq_usage |= BIT(GPIO_INDEX(bank, gpio));
spin_unlock_irqrestore(&bank->lock, flags);
if (type & (IRQ_TYPE_LEVEL_LOW | IRQ_TYPE_LEVEL_HIGH))
pm_runtime_put(bank->dev);
}
+static unsigned int omap_gpio_irq_startup(struct irq_data *d)
+{
+ struct gpio_bank *bank = omap_irq_data_get_bank(d);
+ unsigned int gpio = omap_irq_to_gpio(bank, d->hwirq);
+ unsigned long flags;
+ unsigned offset = GPIO_INDEX(bank, gpio);
+
+ if (!BANK_USED(bank))
+ pm_runtime_get_sync(bank->dev);
+
+ spin_lock_irqsave(&bank->lock, flags);
+ omap_gpio_init_irq(bank, gpio, offset);
+ spin_unlock_irqrestore(&bank->lock, flags);
+ omap_gpio_unmask_irq(d);
+
+ return 0;
+}
+
static void omap_gpio_irq_shutdown(struct irq_data *d)
{
struct gpio_bank *bank = omap_irq_data_get_bank(d);
if (!irqc)
return -ENOMEM;
+ irqc->irq_startup = omap_gpio_irq_startup,
irqc->irq_shutdown = omap_gpio_irq_shutdown,
irqc->irq_ack = omap_gpio_ack_irq,
irqc->irq_mask = omap_gpio_mask_irq,
if (tdev != NULL) {
status = sysfs_create_link(&dev->kobj, &tdev->kobj,
name);
+ put_device(tdev);
} else {
status = -ENODEV;
}
}
status = sysfs_set_active_low(desc, dev, value);
-
+ put_device(dev);
unlock:
mutex_unlock(&sysfs_lock);
* Unconditionally decrement this counter, regardless of the queue's
* type.
*/
- dqm->total_queue_count++;
+ dqm->total_queue_count--;
pr_debug("Total of %d queues are accountable so far\n",
dqm->total_queue_count);
mutex_unlock(&dqm->lock);
}
/* Verify module parameters */
- if ((max_num_of_queues_per_device < 0) ||
+ if ((max_num_of_queues_per_device < 1) ||
(max_num_of_queues_per_device >
KFD_MAX_NUM_OF_QUEUES_PER_DEVICE)) {
- pr_err("kfd: max_num_of_queues_per_device must be between 0 to KFD_MAX_NUM_OF_QUEUES_PER_DEVICE\n");
+ pr_err("kfd: max_num_of_queues_per_device must be between 1 to KFD_MAX_NUM_OF_QUEUES_PER_DEVICE\n");
return -1;
}
BUG_ON(!pqm);
pqn = get_queue_by_qid(pqm, qid);
- BUG_ON(!pqn);
+ if (!pqn) {
+ pr_debug("amdkfd: No queue %d exists for update operation\n",
+ qid);
+ return -EFAULT;
+ }
pqn->q->properties.queue_address = p->queue_address;
pqn->q->properties.queue_size = p->queue_size;
#include "cirrus_drv.h"
int cirrus_modeset = -1;
+int cirrus_bpp = 24;
MODULE_PARM_DESC(modeset, "Disable/Enable modesetting");
module_param_named(modeset, cirrus_modeset, int, 0400);
+MODULE_PARM_DESC(bpp, "Max bits-per-pixel (default:24)");
+module_param_named(bpp, cirrus_bpp, int, 0400);
/*
* This is the generic driver code. This binds the driver to the drm core,
int cirrus_bo_push_sysram(struct cirrus_bo *bo);
int cirrus_bo_pin(struct cirrus_bo *bo, u32 pl_flag, u64 *gpu_addr);
+
+extern int cirrus_bpp;
+
#endif /* __CIRRUS_DRV_H__ */
const int max_pitch = 0x1FF << 3; /* (4096 - 1) & ~111b bytes */
const int max_size = cdev->mc.vram_size;
+ if (bpp > cirrus_bpp)
+ return false;
if (bpp > 32)
return false;
int count;
/* Just add a static list of modes */
- count = drm_add_modes_noedid(connector, 1280, 1024);
- drm_set_preferred_mode(connector, 1024, 768);
+ if (cirrus_bpp <= 24) {
+ count = drm_add_modes_noedid(connector, 1280, 1024);
+ drm_set_preferred_mode(connector, 1024, 768);
+ } else {
+ count = drm_add_modes_noedid(connector, 800, 600);
+ drm_set_preferred_mode(connector, 800, 600);
+ }
return count;
}
}
EXPORT_SYMBOL(drm_fb_helper_add_one_connector);
+static void remove_from_modeset(struct drm_mode_set *set,
+ struct drm_connector *connector)
+{
+ int i, j;
+
+ for (i = 0; i < set->num_connectors; i++) {
+ if (set->connectors[i] == connector)
+ break;
+ }
+
+ if (i == set->num_connectors)
+ return;
+
+ for (j = i + 1; j < set->num_connectors; j++) {
+ set->connectors[j - 1] = set->connectors[j];
+ }
+ set->num_connectors--;
+
+ /* because i915 is pissy about this..
+ * TODO maybe need to makes sure we set it back to !=NULL somewhere?
+ */
+ if (set->num_connectors == 0)
+ set->fb = NULL;
+}
+
int drm_fb_helper_remove_one_connector(struct drm_fb_helper *fb_helper,
struct drm_connector *connector)
{
}
fb_helper->connector_count--;
kfree(fb_helper_connector);
+
+ /* also cleanup dangling references to the connector: */
+ for (i = 0; i < fb_helper->crtc_count; i++)
+ remove_from_modeset(&fb_helper->crtc_info[i].mode_set, connector);
+
return 0;
}
EXPORT_SYMBOL(drm_fb_helper_remove_one_connector);
} else if (id == INTEL_PCH_LPT_DEVICE_ID_TYPE) {
dev_priv->pch_type = PCH_LPT;
DRM_DEBUG_KMS("Found LynxPoint PCH\n");
- WARN_ON(!IS_HASWELL(dev));
- WARN_ON(IS_HSW_ULT(dev));
- } else if (IS_BROADWELL(dev)) {
- dev_priv->pch_type = PCH_LPT;
- dev_priv->pch_id =
- INTEL_PCH_LPT_LP_DEVICE_ID_TYPE;
- DRM_DEBUG_KMS("This is Broadwell, assuming "
- "LynxPoint LP PCH\n");
+ WARN_ON(!IS_HASWELL(dev) && !IS_BROADWELL(dev));
+ WARN_ON(IS_HSW_ULT(dev) || IS_BDW_ULT(dev));
} else if (id == INTEL_PCH_LPT_LP_DEVICE_ID_TYPE) {
dev_priv->pch_type = PCH_LPT;
DRM_DEBUG_KMS("Found LynxPoint LP PCH\n");
- WARN_ON(!IS_HASWELL(dev));
- WARN_ON(!IS_HSW_ULT(dev));
+ WARN_ON(!IS_HASWELL(dev) && !IS_BROADWELL(dev));
+ WARN_ON(!IS_HSW_ULT(dev) && !IS_BDW_ULT(dev));
} else if (id == INTEL_PCH_SPT_DEVICE_ID_TYPE) {
dev_priv->pch_type = PCH_SPT;
DRM_DEBUG_KMS("Found SunrisePoint PCH\n");
#define IS_HSW_EARLY_SDV(dev) (IS_HASWELL(dev) && \
(INTEL_DEVID(dev) & 0xFF00) == 0x0C00)
#define IS_BDW_ULT(dev) (IS_BROADWELL(dev) && \
- ((INTEL_DEVID(dev) & 0xf) == 0x2 || \
- (INTEL_DEVID(dev) & 0xf) == 0x6 || \
+ ((INTEL_DEVID(dev) & 0xf) == 0x6 || \
(INTEL_DEVID(dev) & 0xf) == 0xe))
#define IS_BDW_GT3(dev) (IS_BROADWELL(dev) && \
(INTEL_DEVID(dev) & 0x00F0) == 0x0020)
u32 size = i915_gem_obj_ggtt_size(obj);
uint64_t val;
+ /* Adjust fence size to match tiled area */
+ if (obj->tiling_mode != I915_TILING_NONE) {
+ uint32_t row_size = obj->stride *
+ (obj->tiling_mode == I915_TILING_Y ? 32 : 8);
+ size = (size / row_size) * row_size;
+ }
+
val = (uint64_t)((i915_gem_obj_ggtt_offset(obj) + size - 4096) &
0xfffff000) << 32;
val |= i915_gem_obj_ggtt_offset(obj) & 0xfffff000;
for (i = 0; i < NUM_L3_SLICES(dev); i++)
i915_gem_l3_remap(&dev_priv->ring[RCS], i);
- /*
- * XXX: Contexts should only be initialized once. Doing a switch to the
- * default context switch however is something we'd like to do after
- * reset or thaw (the latter may not actually be necessary for HW, but
- * goes with our code better). Context switching requires rings (for
- * the do_switch), but before enabling PPGTT. So don't move this.
- */
- ret = i915_gem_context_enable(dev_priv);
+ ret = i915_ppgtt_init_hw(dev);
if (ret && ret != -EIO) {
- DRM_ERROR("Context enable failed %d\n", ret);
+ DRM_ERROR("PPGTT enable failed %d\n", ret);
i915_gem_cleanup_ringbuffer(dev);
-
- return ret;
}
- ret = i915_ppgtt_init_hw(dev);
+ ret = i915_gem_context_enable(dev_priv);
if (ret && ret != -EIO) {
- DRM_ERROR("PPGTT enable failed %d\n", ret);
+ DRM_ERROR("Context enable failed %d\n", ret);
i915_gem_cleanup_ringbuffer(dev);
+
+ return ret;
}
return ret;
WARN_ON(panel->backlight.max == 0);
- if (panel->backlight.level == 0) {
+ if (panel->backlight.level <= panel->backlight.min) {
panel->backlight.level = panel->backlight.max;
if (panel->backlight.device)
panel->backlight.device->props.brightness =
static int radeon_benchmark_do_move(struct radeon_device *rdev, unsigned size,
uint64_t saddr, uint64_t daddr,
- int flag, int n)
+ int flag, int n,
+ struct reservation_object *resv)
{
unsigned long start_jiffies;
unsigned long end_jiffies;
case RADEON_BENCHMARK_COPY_DMA:
fence = radeon_copy_dma(rdev, saddr, daddr,
size / RADEON_GPU_PAGE_SIZE,
- NULL);
+ resv);
break;
case RADEON_BENCHMARK_COPY_BLIT:
fence = radeon_copy_blit(rdev, saddr, daddr,
size / RADEON_GPU_PAGE_SIZE,
- NULL);
+ resv);
break;
default:
DRM_ERROR("Unknown copy method\n");
if (rdev->asic->copy.dma) {
time = radeon_benchmark_do_move(rdev, size, saddr, daddr,
- RADEON_BENCHMARK_COPY_DMA, n);
+ RADEON_BENCHMARK_COPY_DMA, n,
+ dobj->tbo.resv);
if (time < 0)
goto out_cleanup;
if (time > 0)
if (rdev->asic->copy.blit) {
time = radeon_benchmark_do_move(rdev, size, saddr, daddr,
- RADEON_BENCHMARK_COPY_BLIT, n);
+ RADEON_BENCHMARK_COPY_BLIT, n,
+ dobj->tbo.resv);
if (time < 0)
goto out_cleanup;
if (time > 0)
if (pll->flags & RADEON_PLL_USE_FRAC_FB_DIV &&
pll->flags & RADEON_PLL_USE_REF_DIV)
ref_div_max = pll->reference_div;
+ else if (pll->flags & RADEON_PLL_PREFER_MINM_OVER_MAXP)
+ /* fix for problems on RS880 */
+ ref_div_max = min(pll->max_ref_div, 7u);
else
ref_div_max = pll->max_ref_div;
struct radeon_bo_va *bo_va;
int r;
- if (rdev->family < CHIP_CAYMAN) {
+ if ((rdev->family < CHIP_CAYMAN) ||
+ (!rdev->accel_working)) {
return 0;
}
struct radeon_bo_va *bo_va;
int r;
- if (rdev->family < CHIP_CAYMAN) {
+ if ((rdev->family < CHIP_CAYMAN) ||
+ (!rdev->accel_working)) {
return;
}
return -ENOMEM;
}
- vm = &fpriv->vm;
- r = radeon_vm_init(rdev, vm);
- if (r) {
- kfree(fpriv);
- return r;
- }
-
if (rdev->accel_working) {
+ vm = &fpriv->vm;
+ r = radeon_vm_init(rdev, vm);
+ if (r) {
+ kfree(fpriv);
+ return r;
+ }
+
r = radeon_bo_reserve(rdev->ring_tmp_bo.bo, false);
if (r) {
radeon_vm_fini(rdev, vm);
radeon_vm_bo_rmv(rdev, vm->ib_bo_va);
radeon_bo_unreserve(rdev->ring_tmp_bo.bo);
}
+ radeon_vm_fini(rdev, vm);
}
- radeon_vm_fini(rdev, vm);
kfree(fpriv);
file_priv->driver_priv = NULL;
}
if (ring == R600_RING_TYPE_DMA_INDEX)
fence = radeon_copy_dma(rdev, gtt_addr, vram_addr,
size / RADEON_GPU_PAGE_SIZE,
- NULL);
+ vram_obj->tbo.resv);
else
fence = radeon_copy_blit(rdev, gtt_addr, vram_addr,
size / RADEON_GPU_PAGE_SIZE,
- NULL);
+ vram_obj->tbo.resv);
if (IS_ERR(fence)) {
DRM_ERROR("Failed GTT->VRAM copy %d\n", i);
r = PTR_ERR(fence);
if (ring == R600_RING_TYPE_DMA_INDEX)
fence = radeon_copy_dma(rdev, vram_addr, gtt_addr,
size / RADEON_GPU_PAGE_SIZE,
- NULL);
+ vram_obj->tbo.resv);
else
fence = radeon_copy_blit(rdev, vram_addr, gtt_addr,
size / RADEON_GPU_PAGE_SIZE,
- NULL);
+ vram_obj->tbo.resv);
if (IS_ERR(fence)) {
DRM_ERROR("Failed VRAM->GTT copy %d\n", i);
r = PTR_ERR(fence);
*/
/* NI is optimized for 256KB fragments, SI and newer for 64KB */
- uint64_t frag_flags = rdev->family == CHIP_CAYMAN ?
+ uint64_t frag_flags = ((rdev->family == CHIP_CAYMAN) ||
+ (rdev->family == CHIP_ARUBA)) ?
R600_PTE_FRAG_256KB : R600_PTE_FRAG_64KB;
- uint64_t frag_align = rdev->family == CHIP_CAYMAN ? 0x200 : 0x80;
+ uint64_t frag_align = ((rdev->family == CHIP_CAYMAN) ||
+ (rdev->family == CHIP_ARUBA)) ? 0x200 : 0x80;
uint64_t frag_start = ALIGN(pe_start, frag_align);
uint64_t frag_end = pe_end & ~(frag_align - 1);
config SENSORS_ADS7828
tristate "Texas Instruments ADS7828 and compatibles"
depends on I2C
+ select REGMAP_I2C
help
If you say yes here you get support for Texas Instruments ADS7828 and
ADS7830 8-channel A/D converters. ADS7828 resolution is 12-bit, while
tristate "Texas Instruments INA219 and compatibles"
depends on I2C
help
- If you say yes here you get support for INA219, INA220, INA226, and
- INA230 power monitor chips.
+ If you say yes here you get support for INA219, INA220, INA226,
+ INA230, and INA231 power monitor chips.
The INA2xx driver is configured for the default configuration of
the part as described in the datasheet.
struct abx500_temp *data = dev_get_drvdata(dev);
struct sensor_device_attribute *attr = to_sensor_dev_attr(devattr);
- return sprintf(buf, "%ld\n", data->min[attr->index]);
+ return sprintf(buf, "%lu\n", data->min[attr->index]);
}
static ssize_t show_max(struct device *dev,
struct abx500_temp *data = dev_get_drvdata(dev);
struct sensor_device_attribute *attr = to_sensor_dev_attr(devattr);
- return sprintf(buf, "%ld\n", data->max[attr->index]);
+ return sprintf(buf, "%lu\n", data->max[attr->index]);
}
static ssize_t show_max_hyst(struct device *dev,
struct abx500_temp *data = dev_get_drvdata(dev);
struct sensor_device_attribute *attr = to_sensor_dev_attr(devattr);
- return sprintf(buf, "%ld\n", data->max_hyst[attr->index]);
+ return sprintf(buf, "%lu\n", data->max_hyst[attr->index]);
}
static ssize_t show_min_alarm(struct device *dev,
#include <linux/err.h>
#include <linux/hwmon.h>
#include <linux/hwmon-sysfs.h>
+#include <linux/bitops.h>
/*
* AD7314 temperature masks
switch (spi_get_device_id(chip->spi_dev)->driver_data) {
case ad7314:
data = (ret & AD7314_TEMP_MASK) >> AD7314_TEMP_SHIFT;
- data = (data << 6) >> 6;
+ data = sign_extend32(data, 9);
return sprintf(buf, "%d\n", 250 * data);
case adt7301:
* register. 1lsb - 31.25 milli degrees centigrade
*/
data = ret & ADT7301_TEMP_MASK;
- data = (data << 2) >> 2;
+ data = sign_extend32(data, 13);
return sprintf(buf, "%d\n",
DIV_ROUND_CLOSEST(data * 3125, 100));
#include <linux/err.h>
#include <linux/regulator/consumer.h>
#include <linux/mutex.h>
+#include <linux/bitops.h>
/* Addresses to scan
* The chip also supports addresses 0x35..0x37. Don't scan those addresses
if (IS_ERR(data))
return PTR_ERR(data);
- temp = (data->temp[index] << 7) >> 7; /* sign extend */
+ temp = sign_extend32(data->temp[index], 8);
return sprintf(buf, "%d\n", temp * 500);/* 0.5 degrees C resolution */
}
#include <linux/hwmon-sysfs.h>
#include <linux/i2c.h>
#include <linux/init.h>
-#include <linux/jiffies.h>
#include <linux/module.h>
-#include <linux/mutex.h>
#include <linux/platform_data/ads7828.h>
+#include <linux/regmap.h>
#include <linux/slab.h>
/* The ADS7828 registers */
-#define ADS7828_NCH 8 /* 8 channels supported */
#define ADS7828_CMD_SD_SE 0x80 /* Single ended inputs */
#define ADS7828_CMD_PD1 0x04 /* Internal vref OFF && A/D ON */
#define ADS7828_CMD_PD3 0x0C /* Internal vref ON && A/D ON */
/* Client specific data */
struct ads7828_data {
- struct i2c_client *client;
- struct mutex update_lock; /* Mutex protecting updates */
- unsigned long last_updated; /* Last updated time (in jiffies) */
- u16 adc_input[ADS7828_NCH]; /* ADS7828_NCH samples */
- bool valid; /* Validity flag */
- bool diff_input; /* Differential input */
- bool ext_vref; /* External voltage reference */
- unsigned int vref_mv; /* voltage reference value */
+ struct regmap *regmap;
u8 cmd_byte; /* Command byte without channel bits */
unsigned int lsb_resol; /* Resolution of the ADC sample LSB */
- s32 (*read_channel)(const struct i2c_client *client, u8 command);
};
/* Command byte C2,C1,C0 - see datasheet */
return cmd | (((ch >> 1) | (ch & 0x01) << 2) << 4);
}
-/* Update data for the device (all 8 channels) */
-static struct ads7828_data *ads7828_update_device(struct device *dev)
-{
- struct ads7828_data *data = dev_get_drvdata(dev);
- struct i2c_client *client = data->client;
-
- mutex_lock(&data->update_lock);
-
- if (time_after(jiffies, data->last_updated + HZ + HZ / 2)
- || !data->valid) {
- unsigned int ch;
- dev_dbg(&client->dev, "Starting ads7828 update\n");
-
- for (ch = 0; ch < ADS7828_NCH; ch++) {
- u8 cmd = ads7828_cmd_byte(data->cmd_byte, ch);
- data->adc_input[ch] = data->read_channel(client, cmd);
- }
- data->last_updated = jiffies;
- data->valid = true;
- }
-
- mutex_unlock(&data->update_lock);
-
- return data;
-}
-
/* sysfs callback function */
static ssize_t ads7828_show_in(struct device *dev, struct device_attribute *da,
char *buf)
{
struct sensor_device_attribute *attr = to_sensor_dev_attr(da);
- struct ads7828_data *data = ads7828_update_device(dev);
- unsigned int value = DIV_ROUND_CLOSEST(data->adc_input[attr->index] *
- data->lsb_resol, 1000);
+ struct ads7828_data *data = dev_get_drvdata(dev);
+ u8 cmd = ads7828_cmd_byte(data->cmd_byte, attr->index);
+ unsigned int regval;
+ int err;
- return sprintf(buf, "%d\n", value);
+ err = regmap_read(data->regmap, cmd, ®val);
+ if (err < 0)
+ return err;
+
+ return sprintf(buf, "%d\n",
+ DIV_ROUND_CLOSEST(regval * data->lsb_resol, 1000));
}
static SENSOR_DEVICE_ATTR(in0_input, S_IRUGO, ads7828_show_in, NULL, 0);
ATTRIBUTE_GROUPS(ads7828);
+static const struct regmap_config ads2828_regmap_config = {
+ .reg_bits = 8,
+ .val_bits = 16,
+};
+
+static const struct regmap_config ads2830_regmap_config = {
+ .reg_bits = 8,
+ .val_bits = 8,
+};
+
static int ads7828_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
struct ads7828_platform_data *pdata = dev_get_platdata(dev);
struct ads7828_data *data;
struct device *hwmon_dev;
+ unsigned int vref_mv = ADS7828_INT_VREF_MV;
+ bool diff_input = false;
+ bool ext_vref = false;
data = devm_kzalloc(dev, sizeof(struct ads7828_data), GFP_KERNEL);
if (!data)
return -ENOMEM;
if (pdata) {
- data->diff_input = pdata->diff_input;
- data->ext_vref = pdata->ext_vref;
- if (data->ext_vref)
- data->vref_mv = pdata->vref_mv;
+ diff_input = pdata->diff_input;
+ ext_vref = pdata->ext_vref;
+ if (ext_vref && pdata->vref_mv)
+ vref_mv = pdata->vref_mv;
}
- /* Bound Vref with min/max values if it was provided */
- if (data->vref_mv)
- data->vref_mv = clamp_val(data->vref_mv,
- ADS7828_EXT_VREF_MV_MIN,
- ADS7828_EXT_VREF_MV_MAX);
- else
- data->vref_mv = ADS7828_INT_VREF_MV;
+ /* Bound Vref with min/max values */
+ vref_mv = clamp_val(vref_mv, ADS7828_EXT_VREF_MV_MIN,
+ ADS7828_EXT_VREF_MV_MAX);
/* ADS7828 uses 12-bit samples, while ADS7830 is 8-bit */
if (id->driver_data == ads7828) {
- data->lsb_resol = DIV_ROUND_CLOSEST(data->vref_mv * 1000, 4096);
- data->read_channel = i2c_smbus_read_word_swapped;
+ data->lsb_resol = DIV_ROUND_CLOSEST(vref_mv * 1000, 4096);
+ data->regmap = devm_regmap_init_i2c(client,
+ &ads2828_regmap_config);
} else {
- data->lsb_resol = DIV_ROUND_CLOSEST(data->vref_mv * 1000, 256);
- data->read_channel = i2c_smbus_read_byte_data;
+ data->lsb_resol = DIV_ROUND_CLOSEST(vref_mv * 1000, 256);
+ data->regmap = devm_regmap_init_i2c(client,
+ &ads2830_regmap_config);
}
- data->cmd_byte = data->ext_vref ? ADS7828_CMD_PD1 : ADS7828_CMD_PD3;
- if (!data->diff_input)
+ data->cmd_byte = ext_vref ? ADS7828_CMD_PD1 : ADS7828_CMD_PD3;
+ if (!diff_input)
data->cmd_byte |= ADS7828_CMD_SD_SE;
- data->client = client;
- mutex_init(&data->update_lock);
-
hwmon_dev = devm_hwmon_device_register_with_groups(dev, client->name,
data,
ads7828_groups);
#include <linux/hwmon-sysfs.h>
#include <linux/jiffies.h>
#include <linux/of.h>
+#include <linux/delay.h>
#include <linux/platform_data/ina2xx.h>
#define INA226_ALERT_LIMIT 0x07
#define INA226_DIE_ID 0xFF
-
/* register count */
#define INA219_REGISTERS 6
#define INA226_REGISTERS 8
/* worst case is 68.10 ms (~14.6Hz, ina219) */
#define INA2XX_CONVERSION_RATE 15
+#define INA2XX_MAX_DELAY 69 /* worst case delay in ms */
+
+#define INA2XX_RSHUNT_DEFAULT 10000
+
+/* bit mask for reading the averaging setting in the configuration register */
+#define INA226_AVG_RD_MASK 0x0E00
+
+#define INA226_READ_AVG(reg) (((reg) & INA226_AVG_RD_MASK) >> 9)
+#define INA226_SHIFT_AVG(val) ((val) << 9)
+
+/* common attrs, ina226 attrs and NULL */
+#define INA2XX_MAX_ATTRIBUTE_GROUPS 3
+
+/*
+ * Both bus voltage and shunt voltage conversion times for ina226 are set
+ * to 0b0100 on POR, which translates to 2200 microseconds in total.
+ */
+#define INA226_TOTAL_CONV_TIME_DEFAULT 2200
enum ina2xx_ids { ina219, ina226 };
struct i2c_client *client;
const struct ina2xx_config *config;
+ long rshunt;
+ u16 curr_config;
+
struct mutex update_lock;
bool valid;
unsigned long last_updated;
+ int update_interval; /* in jiffies */
int kind;
+ const struct attribute_group *groups[INA2XX_MAX_ATTRIBUTE_GROUPS];
u16 regs[INA2XX_MAX_REGISTERS];
};
},
};
-static struct ina2xx_data *ina2xx_update_device(struct device *dev)
+/*
+ * Available averaging rates for ina226. The indices correspond with
+ * the bit values expected by the chip (according to the ina226 datasheet,
+ * table 3 AVG bit settings, found at
+ * http://www.ti.com/lit/ds/symlink/ina226.pdf.
+ */
+static const int ina226_avg_tab[] = { 1, 4, 16, 64, 128, 256, 512, 1024 };
+
+static int ina226_avg_bits(int avg)
+{
+ int i;
+
+ /* Get the closest average from the tab. */
+ for (i = 0; i < ARRAY_SIZE(ina226_avg_tab) - 1; i++) {
+ if (avg <= (ina226_avg_tab[i] + ina226_avg_tab[i + 1]) / 2)
+ break;
+ }
+
+ return i; /* Return 0b0111 for values greater than 1024. */
+}
+
+static int ina226_reg_to_interval(u16 config)
+{
+ int avg = ina226_avg_tab[INA226_READ_AVG(config)];
+
+ /*
+ * Multiply the total conversion time by the number of averages.
+ * Return the result in milliseconds.
+ */
+ return DIV_ROUND_CLOSEST(avg * INA226_TOTAL_CONV_TIME_DEFAULT, 1000);
+}
+
+static u16 ina226_interval_to_reg(int interval, u16 config)
+{
+ int avg, avg_bits;
+
+ avg = DIV_ROUND_CLOSEST(interval * 1000,
+ INA226_TOTAL_CONV_TIME_DEFAULT);
+ avg_bits = ina226_avg_bits(avg);
+
+ return (config & ~INA226_AVG_RD_MASK) | INA226_SHIFT_AVG(avg_bits);
+}
+
+static void ina226_set_update_interval(struct ina2xx_data *data)
+{
+ int ms;
+
+ ms = ina226_reg_to_interval(data->curr_config);
+ data->update_interval = msecs_to_jiffies(ms);
+}
+
+static int ina2xx_calibrate(struct ina2xx_data *data)
+{
+ u16 val = DIV_ROUND_CLOSEST(data->config->calibration_factor,
+ data->rshunt);
+
+ return i2c_smbus_write_word_swapped(data->client,
+ INA2XX_CALIBRATION, val);
+}
+
+/*
+ * Initialize the configuration and calibration registers.
+ */
+static int ina2xx_init(struct ina2xx_data *data)
{
- struct ina2xx_data *data = dev_get_drvdata(dev);
struct i2c_client *client = data->client;
- struct ina2xx_data *ret = data;
+ int ret;
- mutex_lock(&data->update_lock);
+ /* device configuration */
+ ret = i2c_smbus_write_word_swapped(client, INA2XX_CONFIG,
+ data->curr_config);
+ if (ret < 0)
+ return ret;
- if (time_after(jiffies, data->last_updated +
- HZ / INA2XX_CONVERSION_RATE) || !data->valid) {
+ /*
+ * Set current LSB to 1mA, shunt is in uOhms
+ * (equation 13 in datasheet).
+ */
+ return ina2xx_calibrate(data);
+}
- int i;
+static int ina2xx_do_update(struct device *dev)
+{
+ struct ina2xx_data *data = dev_get_drvdata(dev);
+ struct i2c_client *client = data->client;
+ int i, rv, retry;
- dev_dbg(&client->dev, "Starting ina2xx update\n");
+ dev_dbg(&client->dev, "Starting ina2xx update\n");
+ for (retry = 5; retry; retry--) {
/* Read all registers */
for (i = 0; i < data->config->registers; i++) {
- int rv = i2c_smbus_read_word_swapped(client, i);
- if (rv < 0) {
- ret = ERR_PTR(rv);
- goto abort;
- }
+ rv = i2c_smbus_read_word_swapped(client, i);
+ if (rv < 0)
+ return rv;
data->regs[i] = rv;
}
+
+ /*
+ * If the current value in the calibration register is 0, the
+ * power and current registers will also remain at 0. In case
+ * the chip has been reset let's check the calibration
+ * register and reinitialize if needed.
+ */
+ if (data->regs[INA2XX_CALIBRATION] == 0) {
+ dev_warn(dev, "chip not calibrated, reinitializing\n");
+
+ rv = ina2xx_init(data);
+ if (rv < 0)
+ return rv;
+
+ /*
+ * Let's make sure the power and current registers
+ * have been updated before trying again.
+ */
+ msleep(INA2XX_MAX_DELAY);
+ continue;
+ }
+
data->last_updated = jiffies;
data->valid = 1;
+
+ return 0;
}
-abort:
+
+ /*
+ * If we're here then although all write operations succeeded, the
+ * chip still returns 0 in the calibration register. Nothing more we
+ * can do here.
+ */
+ dev_err(dev, "unable to reinitialize the chip\n");
+ return -ENODEV;
+}
+
+static struct ina2xx_data *ina2xx_update_device(struct device *dev)
+{
+ struct ina2xx_data *data = dev_get_drvdata(dev);
+ struct ina2xx_data *ret = data;
+ unsigned long after;
+ int rv;
+
+ mutex_lock(&data->update_lock);
+
+ after = data->last_updated + data->update_interval;
+ if (time_after(jiffies, after) || !data->valid) {
+ rv = ina2xx_do_update(dev);
+ if (rv < 0)
+ ret = ERR_PTR(rv);
+ }
+
mutex_unlock(&data->update_lock);
return ret;
}
/* signed register, LSB=1mA (selected), in mA */
val = (s16)data->regs[reg];
break;
+ case INA2XX_CALIBRATION:
+ val = DIV_ROUND_CLOSEST(data->config->calibration_factor,
+ data->regs[reg]);
+ break;
default:
/* programmer goofed */
WARN_ON_ONCE(1);
ina2xx_get_value(data, attr->index));
}
+static ssize_t ina2xx_set_shunt(struct device *dev,
+ struct device_attribute *da,
+ const char *buf, size_t count)
+{
+ struct ina2xx_data *data = ina2xx_update_device(dev);
+ unsigned long val;
+ int status;
+
+ if (IS_ERR(data))
+ return PTR_ERR(data);
+
+ status = kstrtoul(buf, 10, &val);
+ if (status < 0)
+ return status;
+
+ if (val == 0 ||
+ /* Values greater than the calibration factor make no sense. */
+ val > data->config->calibration_factor)
+ return -EINVAL;
+
+ mutex_lock(&data->update_lock);
+ data->rshunt = val;
+ status = ina2xx_calibrate(data);
+ mutex_unlock(&data->update_lock);
+ if (status < 0)
+ return status;
+
+ return count;
+}
+
+static ssize_t ina226_set_interval(struct device *dev,
+ struct device_attribute *da,
+ const char *buf, size_t count)
+{
+ struct ina2xx_data *data = dev_get_drvdata(dev);
+ unsigned long val;
+ int status;
+
+ status = kstrtoul(buf, 10, &val);
+ if (status < 0)
+ return status;
+
+ if (val > INT_MAX || val == 0)
+ return -EINVAL;
+
+ mutex_lock(&data->update_lock);
+ data->curr_config = ina226_interval_to_reg(val,
+ data->regs[INA2XX_CONFIG]);
+ status = i2c_smbus_write_word_swapped(data->client,
+ INA2XX_CONFIG,
+ data->curr_config);
+
+ ina226_set_update_interval(data);
+ /* Make sure the next access re-reads all registers. */
+ data->valid = 0;
+ mutex_unlock(&data->update_lock);
+ if (status < 0)
+ return status;
+
+ return count;
+}
+
+static ssize_t ina226_show_interval(struct device *dev,
+ struct device_attribute *da, char *buf)
+{
+ struct ina2xx_data *data = ina2xx_update_device(dev);
+
+ if (IS_ERR(data))
+ return PTR_ERR(data);
+
+ /*
+ * We don't use data->update_interval here as we want to display
+ * the actual interval used by the chip and jiffies_to_msecs()
+ * doesn't seem to be accurate enough.
+ */
+ return snprintf(buf, PAGE_SIZE, "%d\n",
+ ina226_reg_to_interval(data->regs[INA2XX_CONFIG]));
+}
+
/* shunt voltage */
static SENSOR_DEVICE_ATTR(in0_input, S_IRUGO, ina2xx_show_value, NULL,
INA2XX_SHUNT_VOLTAGE);
static SENSOR_DEVICE_ATTR(power1_input, S_IRUGO, ina2xx_show_value, NULL,
INA2XX_POWER);
+/* shunt resistance */
+static SENSOR_DEVICE_ATTR(shunt_resistor, S_IRUGO | S_IWUSR,
+ ina2xx_show_value, ina2xx_set_shunt,
+ INA2XX_CALIBRATION);
+
+/* update interval (ina226 only) */
+static SENSOR_DEVICE_ATTR(update_interval, S_IRUGO | S_IWUSR,
+ ina226_show_interval, ina226_set_interval, 0);
+
/* pointers to created device attributes */
static struct attribute *ina2xx_attrs[] = {
&sensor_dev_attr_in0_input.dev_attr.attr,
&sensor_dev_attr_in1_input.dev_attr.attr,
&sensor_dev_attr_curr1_input.dev_attr.attr,
&sensor_dev_attr_power1_input.dev_attr.attr,
+ &sensor_dev_attr_shunt_resistor.dev_attr.attr,
NULL,
};
-ATTRIBUTE_GROUPS(ina2xx);
+
+static const struct attribute_group ina2xx_group = {
+ .attrs = ina2xx_attrs,
+};
+
+static struct attribute *ina226_attrs[] = {
+ &sensor_dev_attr_update_interval.dev_attr.attr,
+ NULL,
+};
+
+static const struct attribute_group ina226_group = {
+ .attrs = ina226_attrs,
+};
static int ina2xx_probe(struct i2c_client *client,
const struct i2c_device_id *id)
struct device *dev = &client->dev;
struct ina2xx_data *data;
struct device *hwmon_dev;
- long shunt = 10000; /* default shunt value 10mOhms */
u32 val;
- int ret;
+ int ret, group = 0;
if (!i2c_check_functionality(adapter, I2C_FUNC_SMBUS_WORD_DATA))
return -ENODEV;
if (dev_get_platdata(dev)) {
pdata = dev_get_platdata(dev);
- shunt = pdata->shunt_uohms;
+ data->rshunt = pdata->shunt_uohms;
} else if (!of_property_read_u32(dev->of_node,
"shunt-resistor", &val)) {
- shunt = val;
+ data->rshunt = val;
+ } else {
+ data->rshunt = INA2XX_RSHUNT_DEFAULT;
}
- if (shunt <= 0)
- return -ENODEV;
-
/* set the device type */
data->kind = id->driver_data;
data->config = &ina2xx_config[data->kind];
-
- /* device configuration */
- ret = i2c_smbus_write_word_swapped(client, INA2XX_CONFIG,
- data->config->config_default);
- if (ret < 0) {
- dev_err(dev,
- "error writing to the config register: %d", ret);
- return -ENODEV;
- }
+ data->curr_config = data->config->config_default;
+ data->client = client;
/*
- * Set current LSB to 1mA, shunt is in uOhms
- * (equation 13 in datasheet).
+ * Ina226 has a variable update_interval. For ina219 we
+ * use a constant value.
*/
- ret = i2c_smbus_write_word_swapped(client, INA2XX_CALIBRATION,
- data->config->calibration_factor / shunt);
+ if (data->kind == ina226)
+ ina226_set_update_interval(data);
+ else
+ data->update_interval = HZ / INA2XX_CONVERSION_RATE;
+
+ if (data->rshunt <= 0 ||
+ data->rshunt > data->config->calibration_factor)
+ return -ENODEV;
+
+ ret = ina2xx_init(data);
if (ret < 0) {
- dev_err(dev,
- "error writing to the calibration register: %d", ret);
+ dev_err(dev, "error configuring the device: %d\n", ret);
return -ENODEV;
}
- data->client = client;
mutex_init(&data->update_lock);
+ data->groups[group++] = &ina2xx_group;
+ if (data->kind == ina226)
+ data->groups[group++] = &ina226_group;
+
hwmon_dev = devm_hwmon_device_register_with_groups(dev, client->name,
- data, ina2xx_groups);
+ data, data->groups);
if (IS_ERR(hwmon_dev))
return PTR_ERR(hwmon_dev);
dev_info(dev, "power monitor %s (Rshunt = %li uOhm)\n",
- id->name, shunt);
+ id->name, data->rshunt);
return 0;
}
{ "ina220", ina219 },
{ "ina226", ina226 },
{ "ina230", ina226 },
+ { "ina231", ina226 },
{ }
};
MODULE_DEVICE_TABLE(i2c, ina2xx_id);
#define JC42_TEMP_MIN 0
#define JC42_TEMP_MAX 125000
-static u16 jc42_temp_to_reg(int temp, bool extended)
+static u16 jc42_temp_to_reg(long temp, bool extended)
{
int ntemp = clamp_val(temp,
extended ? JC42_TEMP_MIN_EXTENDED :
static int jc42_temp_from_reg(s16 reg)
{
- reg &= 0x1fff;
-
- /* sign extend register */
- if (reg & 0x1000)
- reg |= 0xf000;
+ reg = sign_extend32(reg, 12);
/* convert from 0.0625 to 0.001 resolution */
return reg * 125 / 2;
const char *buf, size_t count)
{
struct jc42_data *data = dev_get_drvdata(dev);
- unsigned long val;
+ long val;
int diff, hyst;
int err;
int ret = count;
- if (kstrtoul(buf, 10, &val) < 0)
+ if (kstrtol(buf, 10, &val) < 0)
return -EINVAL;
+ val = clamp_val(val, (data->extended ? JC42_TEMP_MIN_EXTENDED :
+ JC42_TEMP_MIN) - 6000, JC42_TEMP_MAX);
diff = jc42_temp_from_reg(data->temp[t_crit]) - val;
+
hyst = 0;
if (diff > 0) {
if (diff < 2250)
return reg != REG_BANK && reg <= 0x20;
}
-static struct regmap_config nct7802_regmap_config = {
+static const struct regmap_config nct7802_regmap_config = {
.reg_bits = 8,
.val_bits = 8,
.cache_type = REGCACHE_RBTREE,
return 0;
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int tmp102_suspend(struct device *dev)
{
struct i2c_client *client = to_i2c_client(dev);
config &= ~TMP102_CONF_SD;
return i2c_smbus_write_word_swapped(client, TMP102_CONF_REG, config);
}
-
-static const struct dev_pm_ops tmp102_dev_pm_ops = {
- .suspend = tmp102_suspend,
- .resume = tmp102_resume,
-};
-
-#define TMP102_DEV_PM_OPS (&tmp102_dev_pm_ops)
-#else
-#define TMP102_DEV_PM_OPS NULL
#endif /* CONFIG_PM */
+static SIMPLE_DEV_PM_OPS(tmp102_dev_pm_ops, tmp102_suspend, tmp102_resume);
+
static const struct i2c_device_id tmp102_id[] = {
{ "tmp102", 0 },
{ }
static struct i2c_driver tmp102_driver = {
.driver.name = DRIVER_NAME,
- .driver.pm = TMP102_DEV_PM_OPS,
+ .driver.pm = &tmp102_dev_pm_ops,
.probe = tmp102_probe,
.remove = tmp102_remove,
.id_table = tmp102_id,
config I2C_RCAR
tristate "Renesas R-Car I2C Controller"
depends on ARCH_SHMOBILE || COMPILE_TEST
+ select I2C_SLAVE
help
If you say yes to this option, support will be included for the
R-Car I2C controller.
int ret;
pm_runtime_get_sync(&adap->dev);
- clk_prepare_enable(i2c->clk);
+ ret = clk_enable(i2c->clk);
+ if (ret)
+ return ret;
for (retry = 0; retry < adap->retries; retry++) {
ret = s3c24xx_i2c_doxfer(i2c, msgs, num);
if (ret != -EAGAIN) {
- clk_disable_unprepare(i2c->clk);
+ clk_disable(i2c->clk);
pm_runtime_put(&adap->dev);
return ret;
}
udelay(100);
}
- clk_disable_unprepare(i2c->clk);
+ clk_disable(i2c->clk);
pm_runtime_put(&adap->dev);
return -EREMOTEIO;
}
clk_prepare_enable(i2c->clk);
ret = s3c24xx_i2c_init(i2c);
- clk_disable_unprepare(i2c->clk);
+ clk_disable(i2c->clk);
if (ret != 0) {
dev_err(&pdev->dev, "I2C controller init failed\n");
return ret;
i2c->irq = ret = platform_get_irq(pdev, 0);
if (ret <= 0) {
dev_err(&pdev->dev, "cannot find IRQ\n");
+ clk_unprepare(i2c->clk);
return ret;
}
if (ret != 0) {
dev_err(&pdev->dev, "cannot claim IRQ %d\n", i2c->irq);
+ clk_unprepare(i2c->clk);
return ret;
}
}
ret = s3c24xx_i2c_register_cpufreq(i2c);
if (ret < 0) {
dev_err(&pdev->dev, "failed to register cpufreq notifier\n");
+ clk_unprepare(i2c->clk);
return ret;
}
if (ret < 0) {
dev_err(&pdev->dev, "failed to add bus to i2c core\n");
s3c24xx_i2c_deregister_cpufreq(i2c);
+ clk_unprepare(i2c->clk);
return ret;
}
{
struct s3c24xx_i2c *i2c = platform_get_drvdata(pdev);
+ clk_unprepare(i2c->clk);
+
pm_runtime_disable(&i2c->adap.dev);
pm_runtime_disable(&pdev->dev);
{
struct platform_device *pdev = to_platform_device(dev);
struct s3c24xx_i2c *i2c = platform_get_drvdata(pdev);
+ int ret;
if (!IS_ERR(i2c->sysreg))
regmap_write(i2c->sysreg, EXYNOS5_SYS_I2C_CFG, i2c->sys_i2c_cfg);
- clk_prepare_enable(i2c->clk);
+ ret = clk_enable(i2c->clk);
+ if (ret)
+ return ret;
s3c24xx_i2c_init(i2c);
- clk_disable_unprepare(i2c->clk);
+ clk_disable(i2c->clk);
i2c->suspended = 0;
return 0;
int pos;
int sr;
bool send_stop;
+ bool stop_after_dma;
struct resource *res;
struct dma_chan *dma_tx;
if (pd->pos == pd->msg->len) {
/* Send stop if we haven't yet (DMA case) */
- if (pd->send_stop && (iic_rd(pd, ICCR) & ICCR_BBSY))
+ if (pd->send_stop && pd->stop_after_dma)
i2c_op(pd, OP_TX_STOP, 0);
return 1;
}
real_pos = pd->pos - 2;
if (pd->pos == pd->msg->len) {
+ if (pd->stop_after_dma) {
+ /* Simulate PIO end condition after DMA transfer */
+ i2c_op(pd, OP_RX_STOP, 0);
+ pd->pos++;
+ break;
+ }
+
if (real_pos < 0) {
i2c_op(pd, OP_RX_STOP, 0);
break;
sh_mobile_i2c_dma_unmap(pd);
pd->pos = pd->msg->len;
+ pd->stop_after_dma = true;
iic_set_clr(pd, ICIC, 0, ICIC_TDMAE | ICIC_RDMAE);
}
bool do_start = pd->send_stop || !i;
msg = &msgs[i];
pd->send_stop = i == num - 1 || msg->flags & I2C_M_STOP;
+ pd->stop_after_dma = false;
err = start_ch(pd, msg, do_start);
if (err)
}
EXPORT_SYMBOL(i2c_smbus_xfer);
+#if IS_ENABLED(CONFIG_I2C_SLAVE)
int i2c_slave_register(struct i2c_client *client, i2c_slave_cb_t slave_cb)
{
int ret;
return ret;
}
EXPORT_SYMBOL_GPL(i2c_slave_unregister);
+#endif
MODULE_AUTHOR("Simon G. Vogl <simon@tk.uni-linz.ac.at>");
MODULE_DESCRIPTION("I2C-Bus main module");
struct eeprom_data *eeprom;
unsigned long flags;
- if (off + count >= attr->size)
+ if (off + count > attr->size)
return -EFBIG;
eeprom = dev_get_drvdata(container_of(kobj, struct device, kobj));
struct eeprom_data *eeprom;
unsigned long flags;
- if (off + count >= attr->size)
+ if (off + count > attr->size)
return -EFBIG;
eeprom = dev_get_drvdata(container_of(kobj, struct device, kobj));
IB_UVERBS_DECLARE_EX_CMD(create_flow);
IB_UVERBS_DECLARE_EX_CMD(destroy_flow);
-IB_UVERBS_DECLARE_EX_CMD(query_device);
#endif /* UVERBS_H */
return ret;
}
-static void copy_query_dev_fields(struct ib_uverbs_file *file,
- struct ib_uverbs_query_device_resp *resp,
- struct ib_device_attr *attr)
-{
- resp->fw_ver = attr->fw_ver;
- resp->node_guid = file->device->ib_dev->node_guid;
- resp->sys_image_guid = attr->sys_image_guid;
- resp->max_mr_size = attr->max_mr_size;
- resp->page_size_cap = attr->page_size_cap;
- resp->vendor_id = attr->vendor_id;
- resp->vendor_part_id = attr->vendor_part_id;
- resp->hw_ver = attr->hw_ver;
- resp->max_qp = attr->max_qp;
- resp->max_qp_wr = attr->max_qp_wr;
- resp->device_cap_flags = attr->device_cap_flags;
- resp->max_sge = attr->max_sge;
- resp->max_sge_rd = attr->max_sge_rd;
- resp->max_cq = attr->max_cq;
- resp->max_cqe = attr->max_cqe;
- resp->max_mr = attr->max_mr;
- resp->max_pd = attr->max_pd;
- resp->max_qp_rd_atom = attr->max_qp_rd_atom;
- resp->max_ee_rd_atom = attr->max_ee_rd_atom;
- resp->max_res_rd_atom = attr->max_res_rd_atom;
- resp->max_qp_init_rd_atom = attr->max_qp_init_rd_atom;
- resp->max_ee_init_rd_atom = attr->max_ee_init_rd_atom;
- resp->atomic_cap = attr->atomic_cap;
- resp->max_ee = attr->max_ee;
- resp->max_rdd = attr->max_rdd;
- resp->max_mw = attr->max_mw;
- resp->max_raw_ipv6_qp = attr->max_raw_ipv6_qp;
- resp->max_raw_ethy_qp = attr->max_raw_ethy_qp;
- resp->max_mcast_grp = attr->max_mcast_grp;
- resp->max_mcast_qp_attach = attr->max_mcast_qp_attach;
- resp->max_total_mcast_qp_attach = attr->max_total_mcast_qp_attach;
- resp->max_ah = attr->max_ah;
- resp->max_fmr = attr->max_fmr;
- resp->max_map_per_fmr = attr->max_map_per_fmr;
- resp->max_srq = attr->max_srq;
- resp->max_srq_wr = attr->max_srq_wr;
- resp->max_srq_sge = attr->max_srq_sge;
- resp->max_pkeys = attr->max_pkeys;
- resp->local_ca_ack_delay = attr->local_ca_ack_delay;
- resp->phys_port_cnt = file->device->ib_dev->phys_port_cnt;
-}
-
ssize_t ib_uverbs_query_device(struct ib_uverbs_file *file,
const char __user *buf,
int in_len, int out_len)
return ret;
memset(&resp, 0, sizeof resp);
- copy_query_dev_fields(file, &resp, &attr);
+
+ resp.fw_ver = attr.fw_ver;
+ resp.node_guid = file->device->ib_dev->node_guid;
+ resp.sys_image_guid = attr.sys_image_guid;
+ resp.max_mr_size = attr.max_mr_size;
+ resp.page_size_cap = attr.page_size_cap;
+ resp.vendor_id = attr.vendor_id;
+ resp.vendor_part_id = attr.vendor_part_id;
+ resp.hw_ver = attr.hw_ver;
+ resp.max_qp = attr.max_qp;
+ resp.max_qp_wr = attr.max_qp_wr;
+ resp.device_cap_flags = attr.device_cap_flags;
+ resp.max_sge = attr.max_sge;
+ resp.max_sge_rd = attr.max_sge_rd;
+ resp.max_cq = attr.max_cq;
+ resp.max_cqe = attr.max_cqe;
+ resp.max_mr = attr.max_mr;
+ resp.max_pd = attr.max_pd;
+ resp.max_qp_rd_atom = attr.max_qp_rd_atom;
+ resp.max_ee_rd_atom = attr.max_ee_rd_atom;
+ resp.max_res_rd_atom = attr.max_res_rd_atom;
+ resp.max_qp_init_rd_atom = attr.max_qp_init_rd_atom;
+ resp.max_ee_init_rd_atom = attr.max_ee_init_rd_atom;
+ resp.atomic_cap = attr.atomic_cap;
+ resp.max_ee = attr.max_ee;
+ resp.max_rdd = attr.max_rdd;
+ resp.max_mw = attr.max_mw;
+ resp.max_raw_ipv6_qp = attr.max_raw_ipv6_qp;
+ resp.max_raw_ethy_qp = attr.max_raw_ethy_qp;
+ resp.max_mcast_grp = attr.max_mcast_grp;
+ resp.max_mcast_qp_attach = attr.max_mcast_qp_attach;
+ resp.max_total_mcast_qp_attach = attr.max_total_mcast_qp_attach;
+ resp.max_ah = attr.max_ah;
+ resp.max_fmr = attr.max_fmr;
+ resp.max_map_per_fmr = attr.max_map_per_fmr;
+ resp.max_srq = attr.max_srq;
+ resp.max_srq_wr = attr.max_srq_wr;
+ resp.max_srq_sge = attr.max_srq_sge;
+ resp.max_pkeys = attr.max_pkeys;
+ resp.local_ca_ack_delay = attr.local_ca_ack_delay;
+ resp.phys_port_cnt = file->device->ib_dev->phys_port_cnt;
if (copy_to_user((void __user *) (unsigned long) cmd.response,
&resp, sizeof resp))
return ret ? ret : in_len;
}
-
-int ib_uverbs_ex_query_device(struct ib_uverbs_file *file,
- struct ib_udata *ucore,
- struct ib_udata *uhw)
-{
- struct ib_uverbs_ex_query_device_resp resp;
- struct ib_uverbs_ex_query_device cmd;
- struct ib_device_attr attr;
- struct ib_device *device;
- int err;
-
- device = file->device->ib_dev;
- if (ucore->inlen < sizeof(cmd))
- return -EINVAL;
-
- err = ib_copy_from_udata(&cmd, ucore, sizeof(cmd));
- if (err)
- return err;
-
- if (cmd.reserved)
- return -EINVAL;
-
- err = device->query_device(device, &attr);
- if (err)
- return err;
-
- memset(&resp, 0, sizeof(resp));
- copy_query_dev_fields(file, &resp.base, &attr);
- resp.comp_mask = 0;
-
-#ifdef CONFIG_INFINIBAND_ON_DEMAND_PAGING
- if (cmd.comp_mask & IB_USER_VERBS_EX_QUERY_DEVICE_ODP) {
- resp.odp_caps.general_caps = attr.odp_caps.general_caps;
- resp.odp_caps.per_transport_caps.rc_odp_caps =
- attr.odp_caps.per_transport_caps.rc_odp_caps;
- resp.odp_caps.per_transport_caps.uc_odp_caps =
- attr.odp_caps.per_transport_caps.uc_odp_caps;
- resp.odp_caps.per_transport_caps.ud_odp_caps =
- attr.odp_caps.per_transport_caps.ud_odp_caps;
- resp.comp_mask |= IB_USER_VERBS_EX_QUERY_DEVICE_ODP;
- }
-#endif
-
- err = ib_copy_to_udata(ucore, &resp, sizeof(resp));
- if (err)
- return err;
-
- return 0;
-}
struct ib_udata *uhw) = {
[IB_USER_VERBS_EX_CMD_CREATE_FLOW] = ib_uverbs_ex_create_flow,
[IB_USER_VERBS_EX_CMD_DESTROY_FLOW] = ib_uverbs_ex_destroy_flow,
- [IB_USER_VERBS_EX_CMD_QUERY_DEVICE] = ib_uverbs_ex_query_device
};
static void ib_uverbs_add_one(struct ib_device *device);
(1ull << IB_USER_VERBS_CMD_DESTROY_SRQ) |
(1ull << IB_USER_VERBS_CMD_CREATE_XSRQ) |
(1ull << IB_USER_VERBS_CMD_OPEN_QP);
- dev->ib_dev.uverbs_ex_cmd_mask =
- (1ull << IB_USER_VERBS_EX_CMD_QUERY_DEVICE);
dev->ib_dev.query_device = mlx5_ib_query_device;
dev->ib_dev.query_port = mlx5_ib_query_port;
IPOIB_MCAST_FLAG_FOUND = 0, /* used in set_multicast_list */
IPOIB_MCAST_FLAG_SENDONLY = 1,
- /*
- * For IPOIB_MCAST_FLAG_BUSY
- * When set, in flight join and mcast->mc is unreliable
- * When clear and mcast->mc IS_ERR_OR_NULL, need to restart or
- * haven't started yet
- * When clear and mcast->mc is valid pointer, join was successful
- */
- IPOIB_MCAST_FLAG_BUSY = 2,
+ IPOIB_MCAST_FLAG_BUSY = 2, /* joining or already joined */
IPOIB_MCAST_FLAG_ATTACHED = 3,
+ IPOIB_MCAST_JOIN_STARTED = 4,
MAX_SEND_CQE = 16,
IPOIB_CM_COPYBREAK = 256,
struct list_head multicast_list;
struct rb_root multicast_tree;
- struct workqueue_struct *wq;
struct delayed_work mcast_task;
struct work_struct carrier_on_task;
struct work_struct flush_light;
void ipoib_pkey_event(struct work_struct *work);
void ipoib_ib_dev_cleanup(struct net_device *dev);
-int ipoib_ib_dev_open(struct net_device *dev);
+int ipoib_ib_dev_open(struct net_device *dev, int flush);
int ipoib_ib_dev_up(struct net_device *dev);
-int ipoib_ib_dev_down(struct net_device *dev);
-int ipoib_ib_dev_stop(struct net_device *dev);
+int ipoib_ib_dev_down(struct net_device *dev, int flush);
+int ipoib_ib_dev_stop(struct net_device *dev, int flush);
void ipoib_pkey_dev_check_presence(struct net_device *dev);
int ipoib_dev_init(struct net_device *dev, struct ib_device *ca, int port);
void ipoib_mcast_restart_task(struct work_struct *work);
int ipoib_mcast_start_thread(struct net_device *dev);
-int ipoib_mcast_stop_thread(struct net_device *dev);
+int ipoib_mcast_stop_thread(struct net_device *dev, int flush);
void ipoib_mcast_dev_down(struct net_device *dev);
void ipoib_mcast_dev_flush(struct net_device *dev);
}
spin_lock_irq(&priv->lock);
- queue_delayed_work(priv->wq,
+ queue_delayed_work(ipoib_workqueue,
&priv->cm.stale_task, IPOIB_CM_RX_DELAY);
/* Add this entry to passive ids list head, but do not re-add it
* if IB_EVENT_QP_LAST_WQE_REACHED has moved it to flush list. */
spin_lock_irqsave(&priv->lock, flags);
list_splice_init(&priv->cm.rx_drain_list, &priv->cm.rx_reap_list);
ipoib_cm_start_rx_drain(priv);
- queue_work(priv->wq, &priv->cm.rx_reap_task);
+ queue_work(ipoib_workqueue, &priv->cm.rx_reap_task);
spin_unlock_irqrestore(&priv->lock, flags);
} else
ipoib_warn(priv, "cm recv completion event with wrid %d (> %d)\n",
spin_lock_irqsave(&priv->lock, flags);
list_move(&p->list, &priv->cm.rx_reap_list);
spin_unlock_irqrestore(&priv->lock, flags);
- queue_work(priv->wq, &priv->cm.rx_reap_task);
+ queue_work(ipoib_workqueue, &priv->cm.rx_reap_task);
}
return;
}
if (test_and_clear_bit(IPOIB_FLAG_INITIALIZED, &tx->flags)) {
list_move(&tx->list, &priv->cm.reap_list);
- queue_work(priv->wq, &priv->cm.reap_task);
+ queue_work(ipoib_workqueue, &priv->cm.reap_task);
}
clear_bit(IPOIB_FLAG_OPER_UP, &tx->flags);
if (test_and_clear_bit(IPOIB_FLAG_INITIALIZED, &tx->flags)) {
list_move(&tx->list, &priv->cm.reap_list);
- queue_work(priv->wq, &priv->cm.reap_task);
+ queue_work(ipoib_workqueue, &priv->cm.reap_task);
}
spin_unlock_irqrestore(&priv->lock, flags);
tx->dev = dev;
list_add(&tx->list, &priv->cm.start_list);
set_bit(IPOIB_FLAG_INITIALIZED, &tx->flags);
- queue_work(priv->wq, &priv->cm.start_task);
+ queue_work(ipoib_workqueue, &priv->cm.start_task);
return tx;
}
if (test_and_clear_bit(IPOIB_FLAG_INITIALIZED, &tx->flags)) {
spin_lock_irqsave(&priv->lock, flags);
list_move(&tx->list, &priv->cm.reap_list);
- queue_work(priv->wq, &priv->cm.reap_task);
+ queue_work(ipoib_workqueue, &priv->cm.reap_task);
ipoib_dbg(priv, "Reap connection for gid %pI6\n",
tx->neigh->daddr + 4);
tx->neigh = NULL;
skb_queue_tail(&priv->cm.skb_queue, skb);
if (e)
- queue_work(priv->wq, &priv->cm.skb_task);
+ queue_work(ipoib_workqueue, &priv->cm.skb_task);
}
static void ipoib_cm_rx_reap(struct work_struct *work)
}
if (!list_empty(&priv->cm.passive_ids))
- queue_delayed_work(priv->wq,
+ queue_delayed_work(ipoib_workqueue,
&priv->cm.stale_task, IPOIB_CM_RX_DELAY);
spin_unlock_irq(&priv->lock);
}
__ipoib_reap_ah(dev);
if (!test_bit(IPOIB_STOP_REAPER, &priv->flags))
- queue_delayed_work(priv->wq, &priv->ah_reap_task,
+ queue_delayed_work(ipoib_workqueue, &priv->ah_reap_task,
round_jiffies_relative(HZ));
}
drain_tx_cq((struct net_device *)ctx);
}
-int ipoib_ib_dev_open(struct net_device *dev)
+int ipoib_ib_dev_open(struct net_device *dev, int flush)
{
struct ipoib_dev_priv *priv = netdev_priv(dev);
int ret;
}
clear_bit(IPOIB_STOP_REAPER, &priv->flags);
- queue_delayed_work(priv->wq, &priv->ah_reap_task,
+ queue_delayed_work(ipoib_workqueue, &priv->ah_reap_task,
round_jiffies_relative(HZ));
if (!test_and_set_bit(IPOIB_FLAG_INITIALIZED, &priv->flags))
dev_stop:
if (!test_and_set_bit(IPOIB_FLAG_INITIALIZED, &priv->flags))
napi_enable(&priv->napi);
- ipoib_ib_dev_stop(dev);
+ ipoib_ib_dev_stop(dev, flush);
return -1;
}
return ipoib_mcast_start_thread(dev);
}
-int ipoib_ib_dev_down(struct net_device *dev)
+int ipoib_ib_dev_down(struct net_device *dev, int flush)
{
struct ipoib_dev_priv *priv = netdev_priv(dev);
clear_bit(IPOIB_FLAG_OPER_UP, &priv->flags);
netif_carrier_off(dev);
- ipoib_mcast_stop_thread(dev);
+ ipoib_mcast_stop_thread(dev, flush);
ipoib_mcast_dev_flush(dev);
ipoib_flush_paths(dev);
local_bh_enable();
}
-int ipoib_ib_dev_stop(struct net_device *dev)
+int ipoib_ib_dev_stop(struct net_device *dev, int flush)
{
struct ipoib_dev_priv *priv = netdev_priv(dev);
struct ib_qp_attr qp_attr;
/* Wait for all AHs to be reaped */
set_bit(IPOIB_STOP_REAPER, &priv->flags);
cancel_delayed_work(&priv->ah_reap_task);
- flush_workqueue(priv->wq);
+ if (flush)
+ flush_workqueue(ipoib_workqueue);
begin = jiffies;
(unsigned long) dev);
if (dev->flags & IFF_UP) {
- if (ipoib_ib_dev_open(dev)) {
+ if (ipoib_ib_dev_open(dev, 1)) {
ipoib_transport_dev_cleanup(dev);
return -ENODEV;
}
}
if (level >= IPOIB_FLUSH_NORMAL)
- ipoib_ib_dev_down(dev);
+ ipoib_ib_dev_down(dev, 0);
if (level == IPOIB_FLUSH_HEAVY) {
if (test_bit(IPOIB_FLAG_INITIALIZED, &priv->flags))
- ipoib_ib_dev_stop(dev);
- if (ipoib_ib_dev_open(dev) != 0)
+ ipoib_ib_dev_stop(dev, 0);
+ if (ipoib_ib_dev_open(dev, 0) != 0)
return;
if (netif_queue_stopped(dev))
netif_start_queue(dev);
*/
ipoib_flush_paths(dev);
- ipoib_mcast_stop_thread(dev);
+ ipoib_mcast_stop_thread(dev, 1);
ipoib_mcast_dev_flush(dev);
ipoib_transport_dev_cleanup(dev);
set_bit(IPOIB_FLAG_ADMIN_UP, &priv->flags);
- if (ipoib_ib_dev_open(dev)) {
+ if (ipoib_ib_dev_open(dev, 1)) {
if (!test_bit(IPOIB_PKEY_ASSIGNED, &priv->flags))
return 0;
goto err_disable;
return 0;
err_stop:
- ipoib_ib_dev_stop(dev);
+ ipoib_ib_dev_stop(dev, 1);
err_disable:
clear_bit(IPOIB_FLAG_ADMIN_UP, &priv->flags);
netif_stop_queue(dev);
- ipoib_ib_dev_down(dev);
- ipoib_ib_dev_stop(dev);
+ ipoib_ib_dev_down(dev, 1);
+ ipoib_ib_dev_stop(dev, 0);
if (!test_bit(IPOIB_FLAG_SUBINTERFACE, &priv->flags)) {
struct ipoib_dev_priv *cpriv;
return;
}
- queue_work(priv->wq, &priv->restart_task);
+ queue_work(ipoib_workqueue, &priv->restart_task);
}
static u32 ipoib_addr_hash(struct ipoib_neigh_hash *htbl, u8 *daddr)
__ipoib_reap_neigh(priv);
if (!test_bit(IPOIB_STOP_NEIGH_GC, &priv->flags))
- queue_delayed_work(priv->wq, &priv->neigh_reap_task,
+ queue_delayed_work(ipoib_workqueue, &priv->neigh_reap_task,
arp_tbl.gc_interval);
}
/* start garbage collection */
clear_bit(IPOIB_STOP_NEIGH_GC, &priv->flags);
- queue_delayed_work(priv->wq, &priv->neigh_reap_task,
+ queue_delayed_work(ipoib_workqueue, &priv->neigh_reap_task,
arp_tbl.gc_interval);
return 0;
{
struct ipoib_dev_priv *priv = netdev_priv(dev);
+ if (ipoib_neigh_hash_init(priv) < 0)
+ goto out;
/* Allocate RX/TX "rings" to hold queued skbs */
priv->rx_ring = kzalloc(ipoib_recvq_size * sizeof *priv->rx_ring,
GFP_KERNEL);
if (!priv->rx_ring) {
printk(KERN_WARNING "%s: failed to allocate RX ring (%d entries)\n",
ca->name, ipoib_recvq_size);
- goto out;
+ goto out_neigh_hash_cleanup;
}
priv->tx_ring = vzalloc(ipoib_sendq_size * sizeof *priv->tx_ring);
if (ipoib_ib_dev_init(dev, ca, port))
goto out_tx_ring_cleanup;
- /*
- * Must be after ipoib_ib_dev_init so we can allocate a per
- * device wq there and use it here
- */
- if (ipoib_neigh_hash_init(priv) < 0)
- goto out_dev_uninit;
-
return 0;
-out_dev_uninit:
- ipoib_ib_dev_cleanup(dev);
-
out_tx_ring_cleanup:
vfree(priv->tx_ring);
out_rx_ring_cleanup:
kfree(priv->rx_ring);
+out_neigh_hash_cleanup:
+ ipoib_neigh_hash_uninit(dev);
out:
return -ENOMEM;
}
}
unregister_netdevice_many(&head);
- /*
- * Must be before ipoib_ib_dev_cleanup or we delete an in use
- * work queue
- */
- ipoib_neigh_hash_uninit(dev);
-
ipoib_ib_dev_cleanup(dev);
kfree(priv->rx_ring);
priv->rx_ring = NULL;
priv->tx_ring = NULL;
+
+ ipoib_neigh_hash_uninit(dev);
}
static const struct header_ops ipoib_header_ops = {
/* Stop GC if started before flush */
set_bit(IPOIB_STOP_NEIGH_GC, &priv->flags);
cancel_delayed_work(&priv->neigh_reap_task);
- flush_workqueue(priv->wq);
+ flush_workqueue(ipoib_workqueue);
event_failed:
ipoib_dev_cleanup(priv->dev);
/* Stop GC */
set_bit(IPOIB_STOP_NEIGH_GC, &priv->flags);
cancel_delayed_work(&priv->neigh_reap_task);
- flush_workqueue(priv->wq);
+ flush_workqueue(ipoib_workqueue);
unregister_netdev(priv->dev);
free_netdev(priv->dev);
* unregister_netdev() and linkwatch_event take the rtnl lock,
* so flush_scheduled_work() can deadlock during device
* removal.
- *
- * In addition, bringing one device up and another down at the
- * same time can deadlock a single workqueue, so we have this
- * global fallback workqueue, but we also attempt to open a
- * per device workqueue each time we bring an interface up
*/
- ipoib_workqueue = create_singlethread_workqueue("ipoib_flush");
+ ipoib_workqueue = create_singlethread_workqueue("ipoib");
if (!ipoib_workqueue) {
ret = -ENOMEM;
goto err_fs;
spin_unlock_irq(&priv->lock);
priv->tx_wr.wr.ud.remote_qkey = priv->qkey;
set_qkey = 1;
+
+ if (!ipoib_cm_admin_enabled(dev)) {
+ rtnl_lock();
+ dev_set_mtu(dev, min(priv->mcast_mtu, priv->admin_mtu));
+ rtnl_unlock();
+ }
}
if (!test_bit(IPOIB_MCAST_FLAG_SENDONLY, &mcast->flags)) {
struct ipoib_mcast *mcast = multicast->context;
struct net_device *dev = mcast->dev;
- /*
- * We have to take the mutex to force mcast_sendonly_join to
- * return from ib_sa_multicast_join and set mcast->mc to a
- * valid value. Otherwise we were racing with ourselves in
- * that we might fail here, but get a valid return from
- * ib_sa_multicast_join after we had cleared mcast->mc here,
- * resulting in mis-matched joins and leaves and a deadlock
- */
- mutex_lock(&mcast_mutex);
-
/* We trap for port events ourselves. */
if (status == -ENETRESET)
- goto out;
+ return 0;
if (!status)
status = ipoib_mcast_join_finish(mcast, &multicast->rec);
if (status) {
if (mcast->logcount++ < 20)
- ipoib_dbg_mcast(netdev_priv(dev), "sendonly multicast "
- "join failed for %pI6, status %d\n",
+ ipoib_dbg_mcast(netdev_priv(dev), "multicast join failed for %pI6, status %d\n",
mcast->mcmember.mgid.raw, status);
/* Flush out any queued packets */
dev_kfree_skb_any(skb_dequeue(&mcast->pkt_queue));
}
netif_tx_unlock_bh(dev);
+
+ /* Clear the busy flag so we try again */
+ status = test_and_clear_bit(IPOIB_MCAST_FLAG_BUSY,
+ &mcast->flags);
}
-out:
- clear_bit(IPOIB_MCAST_FLAG_BUSY, &mcast->flags);
- if (status)
- mcast->mc = NULL;
- complete(&mcast->done);
- if (status == -ENETRESET)
- status = 0;
- mutex_unlock(&mcast_mutex);
return status;
}
int ret = 0;
if (!test_bit(IPOIB_FLAG_OPER_UP, &priv->flags)) {
- ipoib_dbg_mcast(priv, "device shutting down, no sendonly "
- "multicast joins\n");
+ ipoib_dbg_mcast(priv, "device shutting down, no multicast joins\n");
return -ENODEV;
}
- if (test_bit(IPOIB_MCAST_FLAG_BUSY, &mcast->flags)) {
- ipoib_dbg_mcast(priv, "multicast entry busy, skipping "
- "sendonly join\n");
+ if (test_and_set_bit(IPOIB_MCAST_FLAG_BUSY, &mcast->flags)) {
+ ipoib_dbg_mcast(priv, "multicast entry busy, skipping\n");
return -EBUSY;
}
rec.port_gid = priv->local_gid;
rec.pkey = cpu_to_be16(priv->pkey);
- mutex_lock(&mcast_mutex);
- init_completion(&mcast->done);
- set_bit(IPOIB_MCAST_FLAG_BUSY, &mcast->flags);
mcast->mc = ib_sa_join_multicast(&ipoib_sa_client, priv->ca,
priv->port, &rec,
IB_SA_MCMEMBER_REC_MGID |
if (IS_ERR(mcast->mc)) {
ret = PTR_ERR(mcast->mc);
clear_bit(IPOIB_MCAST_FLAG_BUSY, &mcast->flags);
- complete(&mcast->done);
- ipoib_warn(priv, "ib_sa_join_multicast for sendonly join "
- "failed (ret = %d)\n", ret);
+ ipoib_warn(priv, "ib_sa_join_multicast failed (ret = %d)\n",
+ ret);
} else {
- ipoib_dbg_mcast(priv, "no multicast record for %pI6, starting "
- "sendonly join\n", mcast->mcmember.mgid.raw);
+ ipoib_dbg_mcast(priv, "no multicast record for %pI6, starting join\n",
+ mcast->mcmember.mgid.raw);
}
- mutex_unlock(&mcast_mutex);
return ret;
}
carrier_on_task);
struct ib_port_attr attr;
+ /*
+ * Take rtnl_lock to avoid racing with ipoib_stop() and
+ * turning the carrier back on while a device is being
+ * removed.
+ */
if (ib_query_port(priv->ca, priv->port, &attr) ||
attr.state != IB_PORT_ACTIVE) {
ipoib_dbg(priv, "Keeping carrier off until IB port is active\n");
return;
}
- /*
- * Take rtnl_lock to avoid racing with ipoib_stop() and
- * turning the carrier back on while a device is being
- * removed. However, ipoib_stop() will attempt to flush
- * the workqueue while holding the rtnl lock, so loop
- * on trylock until either we get the lock or we see
- * FLAG_ADMIN_UP go away as that signals that we are bailing
- * and can safely ignore the carrier on work.
- */
- while (!rtnl_trylock()) {
- if (!test_bit(IPOIB_FLAG_ADMIN_UP, &priv->flags))
- return;
- else
- msleep(20);
- }
- if (!ipoib_cm_admin_enabled(priv->dev))
- dev_set_mtu(priv->dev, min(priv->mcast_mtu, priv->admin_mtu));
+ rtnl_lock();
netif_carrier_on(priv->dev);
rtnl_unlock();
}
ipoib_dbg_mcast(priv, "join completion for %pI6 (status %d)\n",
mcast->mcmember.mgid.raw, status);
- /*
- * We have to take the mutex to force mcast_join to
- * return from ib_sa_multicast_join and set mcast->mc to a
- * valid value. Otherwise we were racing with ourselves in
- * that we might fail here, but get a valid return from
- * ib_sa_multicast_join after we had cleared mcast->mc here,
- * resulting in mis-matched joins and leaves and a deadlock
- */
- mutex_lock(&mcast_mutex);
-
/* We trap for port events ourselves. */
- if (status == -ENETRESET)
+ if (status == -ENETRESET) {
+ status = 0;
goto out;
+ }
if (!status)
status = ipoib_mcast_join_finish(mcast, &multicast->rec);
if (!status) {
mcast->backoff = 1;
+ mutex_lock(&mcast_mutex);
if (test_bit(IPOIB_MCAST_RUN, &priv->flags))
- queue_delayed_work(priv->wq, &priv->mcast_task, 0);
+ queue_delayed_work(ipoib_workqueue,
+ &priv->mcast_task, 0);
+ mutex_unlock(&mcast_mutex);
/*
- * Defer carrier on work to priv->wq to avoid a
+ * Defer carrier on work to ipoib_workqueue to avoid a
* deadlock on rtnl_lock here.
*/
if (mcast == priv->broadcast)
- queue_work(priv->wq, &priv->carrier_on_task);
- } else {
- if (mcast->logcount++ < 20) {
- if (status == -ETIMEDOUT || status == -EAGAIN) {
- ipoib_dbg_mcast(priv, "multicast join failed for %pI6, status %d\n",
- mcast->mcmember.mgid.raw, status);
- } else {
- ipoib_warn(priv, "multicast join failed for %pI6, status %d\n",
- mcast->mcmember.mgid.raw, status);
- }
- }
+ queue_work(ipoib_workqueue, &priv->carrier_on_task);
- mcast->backoff *= 2;
- if (mcast->backoff > IPOIB_MAX_BACKOFF_SECONDS)
- mcast->backoff = IPOIB_MAX_BACKOFF_SECONDS;
+ status = 0;
+ goto out;
}
-out:
+
+ if (mcast->logcount++ < 20) {
+ if (status == -ETIMEDOUT || status == -EAGAIN) {
+ ipoib_dbg_mcast(priv, "multicast join failed for %pI6, status %d\n",
+ mcast->mcmember.mgid.raw, status);
+ } else {
+ ipoib_warn(priv, "multicast join failed for %pI6, status %d\n",
+ mcast->mcmember.mgid.raw, status);
+ }
+ }
+
+ mcast->backoff *= 2;
+ if (mcast->backoff > IPOIB_MAX_BACKOFF_SECONDS)
+ mcast->backoff = IPOIB_MAX_BACKOFF_SECONDS;
+
+ /* Clear the busy flag so we try again */
+ status = test_and_clear_bit(IPOIB_MCAST_FLAG_BUSY, &mcast->flags);
+
+ mutex_lock(&mcast_mutex);
spin_lock_irq(&priv->lock);
- clear_bit(IPOIB_MCAST_FLAG_BUSY, &mcast->flags);
- if (status)
- mcast->mc = NULL;
- complete(&mcast->done);
- if (status == -ENETRESET)
- status = 0;
- if (status && test_bit(IPOIB_MCAST_RUN, &priv->flags))
- queue_delayed_work(priv->wq, &priv->mcast_task,
+ if (test_bit(IPOIB_MCAST_RUN, &priv->flags))
+ queue_delayed_work(ipoib_workqueue, &priv->mcast_task,
mcast->backoff * HZ);
spin_unlock_irq(&priv->lock);
mutex_unlock(&mcast_mutex);
-
+out:
+ complete(&mcast->done);
return status;
}
rec.hop_limit = priv->broadcast->mcmember.hop_limit;
}
- mutex_lock(&mcast_mutex);
- init_completion(&mcast->done);
set_bit(IPOIB_MCAST_FLAG_BUSY, &mcast->flags);
+ init_completion(&mcast->done);
+ set_bit(IPOIB_MCAST_JOIN_STARTED, &mcast->flags);
+
mcast->mc = ib_sa_join_multicast(&ipoib_sa_client, priv->ca, priv->port,
&rec, comp_mask, GFP_KERNEL,
ipoib_mcast_join_complete, mcast);
if (mcast->backoff > IPOIB_MAX_BACKOFF_SECONDS)
mcast->backoff = IPOIB_MAX_BACKOFF_SECONDS;
+ mutex_lock(&mcast_mutex);
if (test_bit(IPOIB_MCAST_RUN, &priv->flags))
- queue_delayed_work(priv->wq, &priv->mcast_task,
+ queue_delayed_work(ipoib_workqueue,
+ &priv->mcast_task,
mcast->backoff * HZ);
+ mutex_unlock(&mcast_mutex);
}
- mutex_unlock(&mcast_mutex);
}
void ipoib_mcast_join_task(struct work_struct *work)
ipoib_warn(priv, "failed to allocate broadcast group\n");
mutex_lock(&mcast_mutex);
if (test_bit(IPOIB_MCAST_RUN, &priv->flags))
- queue_delayed_work(priv->wq, &priv->mcast_task,
- HZ);
+ queue_delayed_work(ipoib_workqueue,
+ &priv->mcast_task, HZ);
mutex_unlock(&mcast_mutex);
return;
}
}
if (!test_bit(IPOIB_MCAST_FLAG_ATTACHED, &priv->broadcast->flags)) {
- if (IS_ERR_OR_NULL(priv->broadcast->mc) &&
- !test_bit(IPOIB_MCAST_FLAG_BUSY, &priv->broadcast->flags))
+ if (!test_bit(IPOIB_MCAST_FLAG_BUSY, &priv->broadcast->flags))
ipoib_mcast_join(dev, priv->broadcast, 0);
return;
}
while (1) {
struct ipoib_mcast *mcast = NULL;
- /*
- * Need the mutex so our flags are consistent, need the
- * priv->lock so we don't race with list removals in either
- * mcast_dev_flush or mcast_restart_task
- */
- mutex_lock(&mcast_mutex);
spin_lock_irq(&priv->lock);
list_for_each_entry(mcast, &priv->multicast_list, list) {
- if (IS_ERR_OR_NULL(mcast->mc) &&
- !test_bit(IPOIB_MCAST_FLAG_BUSY, &mcast->flags) &&
- !test_bit(IPOIB_MCAST_FLAG_ATTACHED, &mcast->flags)) {
+ if (!test_bit(IPOIB_MCAST_FLAG_SENDONLY, &mcast->flags)
+ && !test_bit(IPOIB_MCAST_FLAG_BUSY, &mcast->flags)
+ && !test_bit(IPOIB_MCAST_FLAG_ATTACHED, &mcast->flags)) {
/* Found the next unjoined group */
break;
}
}
spin_unlock_irq(&priv->lock);
- mutex_unlock(&mcast_mutex);
if (&mcast->list == &priv->multicast_list) {
/* All done */
break;
}
- if (test_bit(IPOIB_MCAST_FLAG_SENDONLY, &mcast->flags))
- ipoib_mcast_sendonly_join(mcast);
- else
- ipoib_mcast_join(dev, mcast, 1);
+ ipoib_mcast_join(dev, mcast, 1);
return;
}
mutex_lock(&mcast_mutex);
if (!test_and_set_bit(IPOIB_MCAST_RUN, &priv->flags))
- queue_delayed_work(priv->wq, &priv->mcast_task, 0);
+ queue_delayed_work(ipoib_workqueue, &priv->mcast_task, 0);
mutex_unlock(&mcast_mutex);
return 0;
}
-int ipoib_mcast_stop_thread(struct net_device *dev)
+int ipoib_mcast_stop_thread(struct net_device *dev, int flush)
{
struct ipoib_dev_priv *priv = netdev_priv(dev);
cancel_delayed_work(&priv->mcast_task);
mutex_unlock(&mcast_mutex);
- flush_workqueue(priv->wq);
+ if (flush)
+ flush_workqueue(ipoib_workqueue);
return 0;
}
int ret = 0;
if (test_and_clear_bit(IPOIB_MCAST_FLAG_BUSY, &mcast->flags))
- ipoib_warn(priv, "ipoib_mcast_leave on an in-flight join\n");
-
- if (!IS_ERR_OR_NULL(mcast->mc))
ib_sa_free_multicast(mcast->mc);
if (test_and_clear_bit(IPOIB_MCAST_FLAG_ATTACHED, &mcast->flags)) {
memcpy(mcast->mcmember.mgid.raw, mgid, sizeof (union ib_gid));
__ipoib_mcast_add(dev, mcast);
list_add_tail(&mcast->list, &priv->multicast_list);
- if (!test_and_set_bit(IPOIB_MCAST_RUN, &priv->flags))
- queue_delayed_work(priv->wq, &priv->mcast_task, 0);
}
if (!mcast->ah) {
if (test_bit(IPOIB_MCAST_FLAG_BUSY, &mcast->flags))
ipoib_dbg_mcast(priv, "no address vector, "
"but multicast join already started\n");
+ else if (test_bit(IPOIB_MCAST_FLAG_SENDONLY, &mcast->flags))
+ ipoib_mcast_sendonly_join(mcast);
/*
* If lookup completes between here and out:, don't
spin_unlock_irqrestore(&priv->lock, flags);
- /*
- * make sure the in-flight joins have finished before we attempt
- * to leave
- */
+ /* seperate between the wait to the leave*/
list_for_each_entry_safe(mcast, tmcast, &remove_list, list)
- if (test_bit(IPOIB_MCAST_FLAG_BUSY, &mcast->flags))
+ if (test_bit(IPOIB_MCAST_JOIN_STARTED, &mcast->flags))
wait_for_completion(&mcast->done);
list_for_each_entry_safe(mcast, tmcast, &remove_list, list) {
ipoib_dbg_mcast(priv, "restarting multicast task\n");
+ ipoib_mcast_stop_thread(dev, 0);
+
local_irq_save(flags);
netif_addr_lock(dev);
spin_lock(&priv->lock);
netif_addr_unlock(dev);
local_irq_restore(flags);
- /*
- * make sure the in-flight joins have finished before we attempt
- * to leave
- */
- list_for_each_entry_safe(mcast, tmcast, &remove_list, list)
- if (test_bit(IPOIB_MCAST_FLAG_BUSY, &mcast->flags))
- wait_for_completion(&mcast->done);
-
- /*
- * We have to cancel outside of the spinlock, but we have to
- * take the rtnl lock or else we race with the removal of
- * entries from the remove list in mcast_dev_flush as part
- * of ipoib_stop(). We detect the drop of the ADMIN_UP flag
- * to signal that we have hit this particular race, and we
- * return since we know we don't need to do anything else
- * anyway.
- */
- while (!rtnl_trylock()) {
- if (!test_bit(IPOIB_FLAG_ADMIN_UP, &priv->flags))
- return;
- else
- msleep(20);
- }
+ /* We have to cancel outside of the spinlock */
list_for_each_entry_safe(mcast, tmcast, &remove_list, list) {
ipoib_mcast_leave(mcast->dev, mcast);
ipoib_mcast_free(mcast);
}
- /*
- * Restart our join task if needed
- */
- ipoib_mcast_start_thread(dev);
- rtnl_unlock();
+
+ if (test_bit(IPOIB_FLAG_ADMIN_UP, &priv->flags))
+ ipoib_mcast_start_thread(dev);
}
#ifdef CONFIG_INFINIBAND_IPOIB_DEBUG
int ret, size;
int i;
- /*
- * the various IPoIB tasks assume they will never race against
- * themselves, so always use a single thread workqueue
- */
- priv->wq = create_singlethread_workqueue("ipoib_wq");
- if (!priv->wq) {
- printk(KERN_WARNING "ipoib: failed to allocate device WQ\n");
- return -ENODEV;
- }
-
priv->pd = ib_alloc_pd(priv->ca);
if (IS_ERR(priv->pd)) {
printk(KERN_WARNING "%s: failed to allocate PD\n", ca->name);
- goto out_free_wq;
+ return -ENODEV;
}
priv->mr = ib_get_dma_mr(priv->pd, IB_ACCESS_LOCAL_WRITE);
out_free_pd:
ib_dealloc_pd(priv->pd);
-
-out_free_wq:
- destroy_workqueue(priv->wq);
- priv->wq = NULL;
return -ENODEV;
}
if (ib_dealloc_pd(priv->pd))
ipoib_warn(priv, "ib_dealloc_pd failed\n");
-
- if (priv->wq) {
- flush_workqueue(priv->wq);
- destroy_workqueue(priv->wq);
- priv->wq = NULL;
- }
}
void ipoib_event(struct ib_event_handler *handler,
* Asus UX31 0x361f00 20, 15, 0e clickpad
* Asus UX32VD 0x361f02 00, 15, 0e clickpad
* Avatar AVIU-145A2 0x361f00 ? clickpad
+ * Fujitsu LIFEBOOK E544 0x470f00 d0, 12, 09 2 hw buttons
+ * Fujitsu LIFEBOOK E554 0x570f01 40, 14, 0c 2 hw buttons
* Fujitsu H730 0x570f00 c0, 14, 0c 3 hw buttons (**)
* Gigabyte U2442 0x450f01 58, 17, 0c 2 hw buttons
* Lenovo L430 0x350f02 b9, 15, 0c 2 hw buttons (*)
DMI_MATCH(DMI_PRODUCT_NAME, "CELSIUS H730"),
},
},
+ {
+ /* Fujitsu LIFEBOOK E554 does not work with crc_enabled == 0 */
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "FUJITSU"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "LIFEBOOK E554"),
+ },
+ },
+ {
+ /* Fujitsu LIFEBOOK E544 does not work with crc_enabled == 0 */
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "FUJITSU"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "LIFEBOOK E544"),
+ },
+ },
#endif
{ }
};
1232, 5710, 1156, 4696
},
{
- (const char * const []){"LEN0034", "LEN0036", "LEN0039",
- "LEN2002", "LEN2004", NULL},
+ (const char * const []){"LEN0034", "LEN0036", "LEN0037",
+ "LEN0039", "LEN2002", "LEN2004",
+ NULL},
1024, 5112, 2024, 4832
},
{
"LEN0034", /* T431s, L440, L540, T540, W540, X1 Carbon 2nd */
"LEN0035", /* X240 */
"LEN0036", /* T440 */
- "LEN0037",
+ "LEN0037", /* X1 Carbon 2nd */
"LEN0038",
"LEN0039", /* T440s */
"LEN0041",
DMI_MATCH(DMI_PRODUCT_VERSION, "5a"),
},
},
+ {
+ /* Medion Akoya E7225 */
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "Medion"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "Akoya E7225"),
+ DMI_MATCH(DMI_PRODUCT_VERSION, "1.0"),
+ },
+ },
{
/* Blue FB5601 */
.matches = {
}
struct irq_remap_ops amd_iommu_irq_ops = {
- .supported = amd_iommu_supported,
.prepare = amd_iommu_prepare,
.enable = amd_iommu_enable,
.disable = amd_iommu_disable,
/* Make sure ACS will be enabled during PCI probe */
pci_request_acs();
- if (!disable_irq_remap)
- amd_iommu_irq_remap = true;
-
return true;
}
#ifdef CONFIG_IRQ_REMAP
int __init amd_iommu_prepare(void)
{
- return iommu_go_to_state(IOMMU_ACPI_FINISHED);
-}
+ int ret;
-int __init amd_iommu_supported(void)
-{
- return amd_iommu_irq_remap ? 1 : 0;
+ amd_iommu_irq_remap = true;
+
+ ret = iommu_go_to_state(IOMMU_ACPI_FINISHED);
+ if (ret)
+ return ret;
+ return amd_iommu_irq_remap ? 0 : -ENODEV;
}
int __init amd_iommu_enable(void)
extern void amd_iommu_init_api(void);
/* Needed for interrupt remapping */
-extern int amd_iommu_supported(void);
extern int amd_iommu_prepare(void);
extern int amd_iommu_enable(void);
extern void amd_iommu_disable(void);
};
#define IR_X2APIC_MODE(mode) (mode ? (1 << 11) : 0)
-#define IRTE_DEST(dest) ((x2apic_mode) ? dest : dest << 8)
+#define IRTE_DEST(dest) ((eim_mode) ? dest : dest << 8)
+static int __read_mostly eim_mode;
static struct ioapic_scope ir_ioapic[MAX_IO_APICS];
static struct hpet_scope ir_hpet[MAX_HPET_TBS];
if (iommu->ir_table)
return 0;
- ir_table = kzalloc(sizeof(struct ir_table), GFP_ATOMIC);
+ ir_table = kzalloc(sizeof(struct ir_table), GFP_KERNEL);
if (!ir_table)
return -ENOMEM;
- pages = alloc_pages_node(iommu->node, GFP_ATOMIC | __GFP_ZERO,
+ pages = alloc_pages_node(iommu->node, GFP_KERNEL | __GFP_ZERO,
INTR_REMAP_PAGE_ORDER);
if (!pages) {
return dmar->flags & DMAR_X2APIC_OPT_OUT;
}
-static int __init intel_irq_remapping_supported(void)
+static void __init intel_cleanup_irq_remapping(void)
+{
+ struct dmar_drhd_unit *drhd;
+ struct intel_iommu *iommu;
+
+ for_each_iommu(iommu, drhd) {
+ if (ecap_ir_support(iommu->ecap)) {
+ iommu_disable_irq_remapping(iommu);
+ intel_teardown_irq_remapping(iommu);
+ }
+ }
+
+ if (x2apic_supported())
+ pr_warn("Failed to enable irq remapping. You are vulnerable to irq-injection attacks.\n");
+}
+
+static int __init intel_prepare_irq_remapping(void)
{
struct dmar_drhd_unit *drhd;
struct intel_iommu *iommu;
- if (disable_irq_remap)
- return 0;
if (irq_remap_broken) {
printk(KERN_WARNING
"This system BIOS has enabled interrupt remapping\n"
"interrupt remapping is being disabled. Please\n"
"contact your BIOS vendor for an update\n");
add_taint(TAINT_FIRMWARE_WORKAROUND, LOCKDEP_STILL_OK);
- disable_irq_remap = 1;
- return 0;
+ return -ENODEV;
}
+ if (dmar_table_init() < 0)
+ return -ENODEV;
+
if (!dmar_ir_support())
- return 0;
+ return -ENODEV;
+
+ if (parse_ioapics_under_ir() != 1) {
+ printk(KERN_INFO "Not enabling interrupt remapping\n");
+ goto error;
+ }
+ /* First make sure all IOMMUs support IRQ remapping */
for_each_iommu(iommu, drhd)
if (!ecap_ir_support(iommu->ecap))
- return 0;
+ goto error;
- return 1;
+ /* Do the allocations early */
+ for_each_iommu(iommu, drhd)
+ if (intel_setup_irq_remapping(iommu))
+ goto error;
+
+ return 0;
+
+error:
+ intel_cleanup_irq_remapping();
+ return -ENODEV;
}
static int __init intel_enable_irq_remapping(void)
{
struct dmar_drhd_unit *drhd;
struct intel_iommu *iommu;
- bool x2apic_present;
int setup = 0;
int eim = 0;
- x2apic_present = x2apic_supported();
-
- if (parse_ioapics_under_ir() != 1) {
- printk(KERN_INFO "Not enable interrupt remapping\n");
- goto error;
- }
-
- if (x2apic_present) {
- pr_info("Queued invalidation will be enabled to support x2apic and Intr-remapping.\n");
-
+ if (x2apic_supported()) {
eim = !dmar_x2apic_optout();
if (!eim)
printk(KERN_WARNING
/*
* check for the Interrupt-remapping support
*/
- for_each_iommu(iommu, drhd) {
- if (!ecap_ir_support(iommu->ecap))
- continue;
-
+ for_each_iommu(iommu, drhd)
if (eim && !ecap_eim_support(iommu->ecap)) {
printk(KERN_INFO "DRHD %Lx: EIM not supported by DRHD, "
" ecap %Lx\n", drhd->reg_base_addr, iommu->ecap);
- goto error;
+ eim = 0;
}
- }
+ eim_mode = eim;
+ if (eim)
+ pr_info("Queued invalidation will be enabled to support x2apic and Intr-remapping.\n");
/*
* Enable queued invalidation for all the DRHD's.
* Setup Interrupt-remapping for all the DRHD's now.
*/
for_each_iommu(iommu, drhd) {
- if (!ecap_ir_support(iommu->ecap))
- continue;
-
- if (intel_setup_irq_remapping(iommu))
- goto error;
-
iommu_set_irq_remapping(iommu, eim);
setup = 1;
}
return eim ? IRQ_REMAP_X2APIC_MODE : IRQ_REMAP_XAPIC_MODE;
error:
- for_each_iommu(iommu, drhd)
- if (ecap_ir_support(iommu->ecap)) {
- iommu_disable_irq_remapping(iommu);
- intel_teardown_irq_remapping(iommu);
- }
-
- if (x2apic_present)
- pr_warn("Failed to enable irq remapping. You are vulnerable to irq-injection attacks.\n");
-
+ intel_cleanup_irq_remapping();
return -1;
}
}
struct irq_remap_ops intel_irq_remap_ops = {
- .supported = intel_irq_remapping_supported,
- .prepare = dmar_table_init,
+ .prepare = intel_prepare_irq_remapping,
.enable = intel_enable_irq_remapping,
.disable = disable_irq_remapping,
.reenable = reenable_irq_remapping,
#include "irq_remapping.h"
int irq_remapping_enabled;
-
-int disable_irq_remap;
int irq_remap_broken;
int disable_sourceid_checking;
int no_x2apic_optout;
+static int disable_irq_remap;
static struct irq_remap_ops *remap_ops;
static int msi_alloc_remapped_irq(struct pci_dev *pdev, int irq, int nvec);
}
early_param("intremap", setup_irqremap);
-void __init setup_irq_remapping_ops(void)
-{
- remap_ops = &intel_irq_remap_ops;
-
-#ifdef CONFIG_AMD_IOMMU
- if (amd_iommu_irq_ops.prepare() == 0)
- remap_ops = &amd_iommu_irq_ops;
-#endif
-}
-
void set_irq_remapping_broken(void)
{
irq_remap_broken = 1;
}
-int irq_remapping_supported(void)
+int __init irq_remapping_prepare(void)
{
if (disable_irq_remap)
- return 0;
-
- if (!remap_ops || !remap_ops->supported)
- return 0;
-
- return remap_ops->supported();
-}
+ return -ENOSYS;
-int __init irq_remapping_prepare(void)
-{
- if (!remap_ops || !remap_ops->prepare)
- return -ENODEV;
+ if (intel_irq_remap_ops.prepare() == 0)
+ remap_ops = &intel_irq_remap_ops;
+ else if (IS_ENABLED(CONFIG_AMD_IOMMU) &&
+ amd_iommu_irq_ops.prepare() == 0)
+ remap_ops = &amd_iommu_irq_ops;
+ else
+ return -ENOSYS;
- return remap_ops->prepare();
+ return 0;
}
int __init irq_remapping_enable(void)
{
int ret;
- if (!remap_ops || !remap_ops->enable)
+ if (!remap_ops->enable)
return -ENODEV;
ret = remap_ops->enable();
void irq_remapping_disable(void)
{
- if (!irq_remapping_enabled ||
- !remap_ops ||
- !remap_ops->disable)
- return;
-
- remap_ops->disable();
+ if (irq_remapping_enabled && remap_ops->disable)
+ remap_ops->disable();
}
int irq_remapping_reenable(int mode)
{
- if (!irq_remapping_enabled ||
- !remap_ops ||
- !remap_ops->reenable)
- return 0;
+ if (irq_remapping_enabled && remap_ops->reenable)
+ return remap_ops->reenable(mode);
- return remap_ops->reenable(mode);
+ return 0;
}
int __init irq_remap_enable_fault_handling(void)
if (!irq_remapping_enabled)
return 0;
- if (!remap_ops || !remap_ops->enable_faulting)
+ if (!remap_ops->enable_faulting)
return -ENODEV;
return remap_ops->enable_faulting();
unsigned int destination, int vector,
struct io_apic_irq_attr *attr)
{
- if (!remap_ops || !remap_ops->setup_ioapic_entry)
+ if (!remap_ops->setup_ioapic_entry)
return -ENODEV;
return remap_ops->setup_ioapic_entry(irq, entry, destination,
static int set_remapped_irq_affinity(struct irq_data *data,
const struct cpumask *mask, bool force)
{
- if (!config_enabled(CONFIG_SMP) || !remap_ops ||
- !remap_ops->set_affinity)
+ if (!config_enabled(CONFIG_SMP) || !remap_ops->set_affinity)
return 0;
return remap_ops->set_affinity(data, mask, force);
{
struct irq_cfg *cfg = irq_cfg(irq);
- if (!remap_ops || !remap_ops->free_irq)
- return;
-
- if (irq_remapped(cfg))
+ if (irq_remapped(cfg) && remap_ops->free_irq)
remap_ops->free_irq(irq);
}
if (!irq_remapped(cfg))
native_compose_msi_msg(pdev, irq, dest, msg, hpet_id);
- else if (remap_ops && remap_ops->compose_msi_msg)
+ else if (remap_ops->compose_msi_msg)
remap_ops->compose_msi_msg(pdev, irq, dest, msg, hpet_id);
}
static int msi_alloc_remapped_irq(struct pci_dev *pdev, int irq, int nvec)
{
- if (!remap_ops || !remap_ops->msi_alloc_irq)
+ if (!remap_ops->msi_alloc_irq)
return -ENODEV;
return remap_ops->msi_alloc_irq(pdev, irq, nvec);
static int msi_setup_remapped_irq(struct pci_dev *pdev, unsigned int irq,
int index, int sub_handle)
{
- if (!remap_ops || !remap_ops->msi_setup_irq)
+ if (!remap_ops->msi_setup_irq)
return -ENODEV;
return remap_ops->msi_setup_irq(pdev, irq, index, sub_handle);
{
int ret;
- if (!remap_ops || !remap_ops->alloc_hpet_msi)
+ if (!remap_ops->alloc_hpet_msi)
return -ENODEV;
ret = remap_ops->alloc_hpet_msi(irq, id);
struct pci_dev;
struct msi_msg;
-extern int disable_irq_remap;
extern int irq_remap_broken;
extern int disable_sourceid_checking;
extern int no_x2apic_optout;
extern int irq_remapping_enabled;
struct irq_remap_ops {
- /* Check whether Interrupt Remapping is supported */
- int (*supported)(void);
-
/* Initializes hardware and makes it ready for remapping interrupts */
int (*prepare)(void);
#else /* CONFIG_IRQ_REMAP */
#define irq_remapping_enabled 0
-#define disable_irq_remap 1
#define irq_remap_broken 0
#endif /* CONFIG_IRQ_REMAP */
.attach_dev = gart_iommu_attach_dev,
.detach_dev = gart_iommu_detach_dev,
.map = gart_iommu_map,
+ .map_sg = default_iommu_map_sg,
.unmap = gart_iommu_unmap,
.iova_to_phys = gart_iommu_iova_to_phys,
.pgsize_bitmap = GART_IOMMU_PGSIZES,
do_gart_setup(gart, NULL);
gart_handle = gart;
- bus_set_iommu(&platform_bus_type, &gart_iommu_ops);
+
return 0;
}
static int gic_shared_intrs;
static int gic_vpes;
static unsigned int gic_cpu_pin;
+static unsigned int timer_cpu_pin;
static struct irq_chip gic_level_irq_controller, gic_edge_irq_controller;
static void __gic_irq_dispatch(void);
gic_write(GIC_REG(VPE_OTHER, GIC_VPE_COMPARE_MAP), val);
break;
case GIC_LOCAL_INT_TIMER:
+ /* CONFIG_MIPS_CMP workaround (see __gic_init) */
+ val = GIC_MAP_TO_PIN_MSK | timer_cpu_pin;
gic_write(GIC_REG(VPE_OTHER, GIC_VPE_TIMER_MAP), val);
break;
case GIC_LOCAL_INT_PERFCTR:
if (cpu_has_veic) {
/* Always use vector 1 in EIC mode */
gic_cpu_pin = 0;
+ timer_cpu_pin = gic_cpu_pin;
set_vi_handler(gic_cpu_pin + GIC_PIN_TO_VEC_OFFSET,
__gic_irq_dispatch);
} else {
gic_cpu_pin = cpu_vec - GIC_CPU_PIN_OFFSET;
irq_set_chained_handler(MIPS_CPU_IRQ_BASE + cpu_vec,
gic_irq_dispatch);
+ /*
+ * With the CMP implementation of SMP (deprecated), other CPUs
+ * are started by the bootloader and put into a timer based
+ * waiting poll loop. We must not re-route those CPU's local
+ * timer interrupts as the wait instruction will never finish,
+ * so just handle whatever CPU interrupt it is routed to by
+ * default.
+ *
+ * This workaround should be removed when CMP support is
+ * dropped.
+ */
+ if (IS_ENABLED(CONFIG_MIPS_CMP) &&
+ gic_local_irq_is_routable(GIC_LOCAL_INT_TIMER)) {
+ timer_cpu_pin = gic_read(GIC_REG(VPE_LOCAL,
+ GIC_VPE_TIMER_MAP)) &
+ GIC_MAP_MSK;
+ irq_set_chained_handler(MIPS_CPU_IRQ_BASE +
+ GIC_CPU_PIN_OFFSET +
+ timer_cpu_pin,
+ gic_irq_dispatch);
+ } else {
+ timer_cpu_pin = gic_cpu_pin;
+ }
}
gic_irq_domain = irq_domain_add_simple(node, GIC_NUM_LOCAL_INTRS +
add_ai(plci, &parms[5]);
sig_req(plci, REJECT, 0);
}
- else if (Reject == 1 || Reject > 9)
+ else if (Reject == 1 || Reject >= 9)
{
add_ai(plci, &parms[5]);
sig_req(plci, HANGUP, 0);
menuconfig MD
bool "Multiple devices driver support (RAID and LVM)"
depends on BLOCK
+ select SRCU
help
Support multiple physical spindles through a single logical device.
Required for RAID and logical volume management.
/* this page has not been allocated yet */
spin_unlock_irq(&bitmap->lock);
+ /* It is possible that this is being called inside a
+ * prepare_to_wait/finish_wait loop from raid5c:make_request().
+ * In general it is not permitted to sleep in that context as it
+ * can cause the loop to spin freely.
+ * That doesn't apply here as we can only reach this point
+ * once with any loop.
+ * When this function completes, either bp[page].map or
+ * bp[page].hijacked. In either case, this function will
+ * abort before getting to this point again. So there is
+ * no risk of a free-spin, and so it is safe to assert
+ * that sleeping here is allowed.
+ */
+ sched_annotate_sleep();
mappage = kzalloc(PAGE_SIZE, GFP_NOIO);
spin_lock_irq(&bitmap->lock);
cmd = kzalloc(sizeof(*cmd), GFP_KERNEL);
if (!cmd) {
DMERR("could not allocate metadata struct");
- return NULL;
+ return ERR_PTR(-ENOMEM);
}
atomic_set(&cmd->ref_count, 1);
return cmd;
cmd = metadata_open(bdev, data_block_size, may_format_device, policy_hint_size);
- if (cmd) {
+ if (!IS_ERR(cmd)) {
mutex_lock(&table_lock);
cmd2 = lookup(bdev);
if (cmd2) {
{
struct dm_cache_metadata *cmd = lookup_or_open(bdev, data_block_size,
may_format_device, policy_hint_size);
- if (cmd && !same_params(cmd, data_block_size)) {
+
+ if (!IS_ERR(cmd) && !same_params(cmd, data_block_size)) {
dm_cache_metadata_close(cmd);
- return NULL;
+ return ERR_PTR(-EINVAL);
}
return cmd;
struct pool_c *pt = ti->private;
struct pool *pool = pt->pool;
+ if (get_pool_mode(pool) >= PM_READ_ONLY) {
+ DMERR("%s: unable to service pool target messages in READ_ONLY or FAIL mode",
+ dm_device_name(pool->pool_md));
+ return -EINVAL;
+ }
+
if (!strcasecmp(argv[0], "create_thin"))
r = process_create_thin_mesg(argc, argv, pool);
(unsigned long long)sh->sector,
rcw, qread, test_bit(STRIPE_DELAYED, &sh->state));
}
+
+ if (rcw > disks && rmw > disks &&
+ !test_bit(STRIPE_PREREAD_ACTIVE, &sh->state))
+ set_bit(STRIPE_DELAYED, &sh->state);
+
/* now if nothing is locked, and if we have enough data,
* we can start a write request
*/
config NETPOLL
def_bool NETCONSOLE
+ select SRCU
config NET_POLL_CONTROLLER
def_bool NETPOLL
cfhsi = netdev_priv(dev);
cfhsi_netlink_parms(data, cfhsi);
- dev_net_set(cfhsi->ndev, src_net);
get_ops = symbol_get(cfhsi_get_ops);
if (!get_ops) {
config LANCE
tristate "AMD LANCE and PCnet (AT1500 and NE2100) support"
- depends on ISA && ISA_DMA_API
+ depends on ISA && ISA_DMA_API && !ARM
---help---
If you have a network (Ethernet) card of this type, say Y and read
the Ethernet-HOWTO, available from
config NI65
tristate "NI6510 support"
- depends on ISA && ISA_DMA_API
+ depends on ISA && ISA_DMA_API && !ARM
---help---
If you have a network (Ethernet) card of this type, say Y and read
the Ethernet-HOWTO, available from
do {
/* WARNING: MACE_IR is a READ/CLEAR port! */
status = inb(ioaddr + AM2150_MACE_BASE + MACE_IR);
+ if (!(status & ~MACE_IMR_DEFAULT) && IntrCnt == MACE_MAX_IR_ITERATIONS)
+ return IRQ_NONE;
pr_debug("mace_interrupt: irq 0x%X status 0x%X.\n", irq, status);
hw_feat->sph = XGMAC_GET_BITS(mac_hfr1, MAC_HWF1R, SPHEN);
hw_feat->tso = XGMAC_GET_BITS(mac_hfr1, MAC_HWF1R, TSOEN);
hw_feat->dma_debug = XGMAC_GET_BITS(mac_hfr1, MAC_HWF1R, DBGMEMA);
+ hw_feat->rss = XGMAC_GET_BITS(mac_hfr1, MAC_HWF1R, RSSEN);
hw_feat->tc_cnt = XGMAC_GET_BITS(mac_hfr1, MAC_HWF1R, NUMTC);
hw_feat->hash_table_size = XGMAC_GET_BITS(mac_hfr1, MAC_HWF1R,
HASHTBLSZ);
break;
}
- /* The Queue and Channel counts are zero based so increment them
+ /* The Queue, Channel and TC counts are zero based so increment them
* to get the actual number
*/
hw_feat->rx_q_cnt++;
hw_feat->tx_q_cnt++;
hw_feat->rx_ch_cnt++;
hw_feat->tx_ch_cnt++;
+ hw_feat->tc_cnt++;
DBGPR("<--xgbe_get_all_hw_features\n");
}
if (unlikely(xgene_enet_is_desc_slot_empty(raw_desc)))
break;
+ /* read fpqnum field after dataaddr field */
+ dma_rmb();
if (is_rx_desc(raw_desc))
ret = xgene_enet_rx_frame(ring, raw_desc);
else
will be called cs89x0.
config CS89x0_PLATFORM
- bool "CS89x0 platform driver support"
+ bool "CS89x0 platform driver support" if HAS_IOPORT_MAP
+ default !HAS_IOPORT_MAP
depends on CS89x0
help
Say Y to compile the cs89x0 driver as a platform driver. This
return -EBUSY;
/* Fill regular entries */
- for (; i < MAX_FILER_IDX - 1 && (tab->fe[i].ctrl | tab->fe[i].ctrl);
+ for (; i < MAX_FILER_IDX - 1 && (tab->fe[i].ctrl | tab->fe[i].prop);
i++)
gfar_write_filer(priv, i, tab->fe[i].ctrl, tab->fe[i].prop);
/* Fill the rest with fall-troughs */
static int igbvf_tso(struct igbvf_adapter *adapter,
struct igbvf_ring *tx_ring,
- struct sk_buff *skb, u32 tx_flags, u8 *hdr_len)
+ struct sk_buff *skb, u32 tx_flags, u8 *hdr_len,
+ __be16 protocol)
{
struct e1000_adv_tx_context_desc *context_desc;
struct igbvf_buffer *buffer_info;
l4len = tcp_hdrlen(skb);
*hdr_len += l4len;
- if (skb->protocol == htons(ETH_P_IP)) {
+ if (protocol == htons(ETH_P_IP)) {
struct iphdr *iph = ip_hdr(skb);
iph->tot_len = 0;
iph->check = 0;
/* ADV DTYP TUCMD MKRLOC/ISCSIHEDLEN */
tu_cmd |= (E1000_TXD_CMD_DEXT | E1000_ADVTXD_DTYP_CTXT);
- if (skb->protocol == htons(ETH_P_IP))
+ if (protocol == htons(ETH_P_IP))
tu_cmd |= E1000_ADVTXD_TUCMD_IPV4;
tu_cmd |= E1000_ADVTXD_TUCMD_L4T_TCP;
static inline bool igbvf_tx_csum(struct igbvf_adapter *adapter,
struct igbvf_ring *tx_ring,
- struct sk_buff *skb, u32 tx_flags)
+ struct sk_buff *skb, u32 tx_flags,
+ __be16 protocol)
{
struct e1000_adv_tx_context_desc *context_desc;
unsigned int i;
tu_cmd |= (E1000_TXD_CMD_DEXT | E1000_ADVTXD_DTYP_CTXT);
if (skb->ip_summed == CHECKSUM_PARTIAL) {
- switch (skb->protocol) {
+ switch (protocol) {
case htons(ETH_P_IP):
tu_cmd |= E1000_ADVTXD_TUCMD_IPV4;
if (ip_hdr(skb)->protocol == IPPROTO_TCP)
u8 hdr_len = 0;
int count = 0;
int tso = 0;
+ __be16 protocol = vlan_get_protocol(skb);
if (test_bit(__IGBVF_DOWN, &adapter->state)) {
dev_kfree_skb_any(skb);
tx_flags |= (vlan_tx_tag_get(skb) << IGBVF_TX_FLAGS_VLAN_SHIFT);
}
- if (skb->protocol == htons(ETH_P_IP))
+ if (protocol == htons(ETH_P_IP))
tx_flags |= IGBVF_TX_FLAGS_IPV4;
first = tx_ring->next_to_use;
tso = skb_is_gso(skb) ?
- igbvf_tso(adapter, tx_ring, skb, tx_flags, &hdr_len) : 0;
+ igbvf_tso(adapter, tx_ring, skb, tx_flags, &hdr_len, protocol) : 0;
if (unlikely(tso < 0)) {
dev_kfree_skb_any(skb);
return NETDEV_TX_OK;
if (tso)
tx_flags |= IGBVF_TX_FLAGS_TSO;
- else if (igbvf_tx_csum(adapter, tx_ring, skb, tx_flags) &&
+ else if (igbvf_tx_csum(adapter, tx_ring, skb, tx_flags, protocol) &&
(skb->ip_summed == CHECKSUM_PARTIAL))
tx_flags |= IGBVF_TX_FLAGS_CSUM;
if (!vhdr)
goto out_drop;
- protocol = vhdr->h_vlan_encapsulated_proto;
tx_flags |= ntohs(vhdr->h_vlan_TCI) <<
IXGBE_TX_FLAGS_VLAN_SHIFT;
tx_flags |= IXGBE_TX_FLAGS_SW_VLAN;
}
+ protocol = vlan_get_protocol(skb);
if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP) &&
adapter->ptp_clock &&
/* ADV DTYP TUCMD MKRLOC/ISCSIHEDLEN */
type_tucmd = IXGBE_ADVTXD_TUCMD_L4T_TCP;
- if (skb->protocol == htons(ETH_P_IP)) {
+ if (first->protocol == htons(ETH_P_IP)) {
struct iphdr *iph = ip_hdr(skb);
iph->tot_len = 0;
iph->check = 0;
if (skb->ip_summed == CHECKSUM_PARTIAL) {
u8 l4_hdr = 0;
- switch (skb->protocol) {
+ switch (first->protocol) {
case htons(ETH_P_IP):
vlan_macip_lens |= skb_network_header_len(skb);
type_tucmd |= IXGBE_ADVTXD_TUCMD_IPV4;
extern int mlx4_log_num_mgm_entry_size;
extern int log_mtts_per_seg;
-#define MLX4_MAX_NUM_SLAVES (MLX4_MAX_NUM_PF + MLX4_MAX_NUM_VF)
+#define MLX4_MAX_NUM_SLAVES (min(MLX4_MAX_NUM_PF + MLX4_MAX_NUM_VF, \
+ MLX4_MFUNC_MAX))
#define ALL_SLAVES 0xff
struct mlx4_bitmap {
tx_complete = qlcnic_process_cmd_ring(adapter, tx_ring,
budget);
work_done = qlcnic_process_rcv_ring(sds_ring, budget);
- if ((work_done < budget) && tx_complete) {
+
+ /* Check if we need a repoll */
+ if (!tx_complete)
+ work_done = budget;
+
+ if (work_done < budget) {
napi_complete(&sds_ring->napi);
if (test_bit(__QLCNIC_DEV_UP, &adapter->state)) {
qlcnic_enable_sds_intr(adapter, sds_ring);
napi_complete(&tx_ring->napi);
if (test_bit(__QLCNIC_DEV_UP, &adapter->state))
qlcnic_enable_tx_intr(adapter, tx_ring);
+ } else {
+ /* As qlcnic_process_cmd_ring() returned 0, we need a repoll*/
+ work_done = budget;
}
return work_done;
tx_complete = qlcnic_process_cmd_ring(adapter, tx_ring, budget);
work_done = qlcnic_83xx_process_rcv_ring(sds_ring, budget);
- if ((work_done < budget) && tx_complete) {
+
+ /* Check if we need a repoll */
+ if (!tx_complete)
+ work_done = budget;
+
+ if (work_done < budget) {
napi_complete(&sds_ring->napi);
qlcnic_enable_sds_intr(adapter, sds_ring);
}
tx_complete = qlcnic_process_cmd_ring(adapter, tx_ring, budget);
work_done = qlcnic_83xx_process_rcv_ring(sds_ring, budget);
- if ((work_done < budget) && tx_complete) {
+
+ /* Check if we need a repoll */
+ if (!tx_complete)
+ work_done = budget;
+
+ if (work_done < budget) {
napi_complete(&sds_ring->napi);
qlcnic_enable_sds_intr(adapter, sds_ring);
}
napi_complete(&tx_ring->napi);
if (test_bit(__QLCNIC_DEV_UP , &adapter->state))
qlcnic_enable_tx_intr(adapter, tx_ring);
+ } else {
+ /* need a repoll */
+ work_done = budget;
}
return work_done;
{
struct ql_adapter *qdev = netdev_priv(ndev);
int status = 0;
+ bool need_restart = netif_running(ndev);
- status = ql_adapter_down(qdev);
- if (status) {
- netif_err(qdev, link, qdev->ndev,
- "Failed to bring down the adapter\n");
- return status;
+ if (need_restart) {
+ status = ql_adapter_down(qdev);
+ if (status) {
+ netif_err(qdev, link, qdev->ndev,
+ "Failed to bring down the adapter\n");
+ return status;
+ }
}
/* update the features with resent change */
ndev->features = features;
- status = ql_adapter_up(qdev);
- if (status) {
- netif_err(qdev, link, qdev->ndev,
- "Failed to bring up the adapter\n");
- return status;
+ if (need_restart) {
+ status = ql_adapter_up(qdev);
+ if (status) {
+ netif_err(qdev, link, qdev->ndev,
+ "Failed to bring up the adapter\n");
+ return status;
+ }
}
+
return status;
}
skb_shinfo(nskb)->gso_size = skb_shinfo(skb)->gso_size;
skb_shinfo(nskb)->gso_type = skb_shinfo(skb)->gso_type;
}
+ nskb->queue_mapping = skb->queue_mapping;
dev_kfree_skb(skb);
skb = nskb;
}
u64 req_id;
unsigned int section_index = NETVSC_INVALID_INDEX;
u32 msg_size = 0;
- struct sk_buff *skb;
+ struct sk_buff *skb = NULL;
u16 q_idx = packet->q_idx;
packet);
skb = (struct sk_buff *)
(unsigned long)packet->send_completion_tid;
- if (skb)
- dev_kfree_skb_any(skb);
packet->page_buf_cnt = 0;
}
}
packet, ret);
}
+ if (ret != 0) {
+ if (section_index != NETVSC_INVALID_INDEX)
+ netvsc_free_send_slot(net_device, section_index);
+ } else if (skb) {
+ dev_kfree_skb_any(skb);
+ }
+
return ret;
}
#include <linux/fs.h>
#include <linux/uio.h>
-#include <net/ipv6.h>
#include <net/net_namespace.h>
#include <net/rtnetlink.h>
#include <net/sock.h>
static const struct proto_ops macvtap_socket_ops;
#define TUN_OFFLOADS (NETIF_F_HW_CSUM | NETIF_F_TSO_ECN | NETIF_F_TSO | \
- NETIF_F_TSO6)
+ NETIF_F_TSO6 | NETIF_F_UFO)
#define RX_OFFLOADS (NETIF_F_GRO | NETIF_F_LRO)
#define TAP_FEATURES (NETIF_F_GSO | NETIF_F_SG)
gso_type = SKB_GSO_TCPV6;
break;
case VIRTIO_NET_HDR_GSO_UDP:
- pr_warn_once("macvtap: %s: using disabled UFO feature; please fix this program\n",
- current->comm);
gso_type = SKB_GSO_UDP;
- if (skb->protocol == htons(ETH_P_IPV6))
- ipv6_proxy_select_ident(skb);
break;
default:
return -EINVAL;
vnet_hdr->gso_type = VIRTIO_NET_HDR_GSO_TCPV4;
else if (sinfo->gso_type & SKB_GSO_TCPV6)
vnet_hdr->gso_type = VIRTIO_NET_HDR_GSO_TCPV6;
+ else if (sinfo->gso_type & SKB_GSO_UDP)
+ vnet_hdr->gso_type = VIRTIO_NET_HDR_GSO_UDP;
else
BUG();
if (sinfo->gso_type & SKB_GSO_TCP_ECN)
if (arg & TUN_F_TSO6)
feature_mask |= NETIF_F_TSO6;
}
+
+ if (arg & TUN_F_UFO)
+ feature_mask |= NETIF_F_UFO;
}
/* tun/tap driver inverts the usage for TSO offloads, where
* When user space turns off TSO, we turn off GSO/LRO so that
* user-space will not receive TSO frames.
*/
- if (feature_mask & (NETIF_F_TSO | NETIF_F_TSO6))
+ if (feature_mask & (NETIF_F_TSO | NETIF_F_TSO6 | NETIF_F_UFO))
features |= RX_OFFLOADS;
else
features &= ~RX_OFFLOADS;
case TUNSETOFFLOAD:
/* let the user check for future flags */
if (arg & ~(TUN_F_CSUM | TUN_F_TSO4 | TUN_F_TSO6 |
- TUN_F_TSO_ECN))
+ TUN_F_TSO_ECN | TUN_F_UFO))
return -EINVAL;
rtnl_lock();
/*
* See if we managed to reduce the size of the packet.
*/
- if (olen < isize) {
+ if (olen < isize && olen <= osize) {
state->stats.comp_bytes += olen;
state->stats.comp_packets++;
} else {
#include <linux/nsproxy.h>
#include <linux/virtio_net.h>
#include <linux/rcupdate.h>
-#include <net/ipv6.h>
#include <net/net_namespace.h>
#include <net/netns/generic.h>
#include <net/rtnetlink.h>
struct net_device *dev;
netdev_features_t set_features;
#define TUN_USER_FEATURES (NETIF_F_HW_CSUM|NETIF_F_TSO_ECN|NETIF_F_TSO| \
- NETIF_F_TSO6)
+ NETIF_F_TSO6|NETIF_F_UFO)
int vnet_hdr_sz;
int sndbuf;
break;
}
- skb_reset_network_header(skb);
-
if (gso.gso_type != VIRTIO_NET_HDR_GSO_NONE) {
pr_debug("GSO!\n");
switch (gso.gso_type & ~VIRTIO_NET_HDR_GSO_ECN) {
skb_shinfo(skb)->gso_type = SKB_GSO_TCPV6;
break;
case VIRTIO_NET_HDR_GSO_UDP:
- {
- static bool warned;
-
- if (!warned) {
- warned = true;
- netdev_warn(tun->dev,
- "%s: using disabled UFO feature; please fix this program\n",
- current->comm);
- }
skb_shinfo(skb)->gso_type = SKB_GSO_UDP;
- if (skb->protocol == htons(ETH_P_IPV6))
- ipv6_proxy_select_ident(skb);
break;
- }
default:
tun->dev->stats.rx_frame_errors++;
kfree_skb(skb);
skb_shinfo(skb)->tx_flags |= SKBTX_SHARED_FRAG;
}
+ skb_reset_network_header(skb);
skb_probe_transport_header(skb, 0);
rxhash = skb_get_hash(skb);
gso.gso_type = VIRTIO_NET_HDR_GSO_TCPV4;
else if (sinfo->gso_type & SKB_GSO_TCPV6)
gso.gso_type = VIRTIO_NET_HDR_GSO_TCPV6;
+ else if (sinfo->gso_type & SKB_GSO_UDP)
+ gso.gso_type = VIRTIO_NET_HDR_GSO_UDP;
else {
pr_err("unexpected GSO type: "
"0x%x, gso_size %d, hdr_len %d\n",
features |= NETIF_F_TSO6;
arg &= ~(TUN_F_TSO4|TUN_F_TSO6);
}
+
+ if (arg & TUN_F_UFO) {
+ features |= NETIF_F_UFO;
+ arg &= ~TUN_F_UFO;
+ }
}
/* This gives the user a way to test for new features in future by
int ret;
udelay(1);
- ret = sr_read_reg(dev, EPCR, &tmp);
+ ret = sr_read_reg(dev, SR_EPCR, &tmp);
if (ret < 0)
return ret;
mutex_lock(&dev->phy_mutex);
- sr_write_reg(dev, EPAR, phy ? (reg | EPAR_PHY_ADR) : reg);
- sr_write_reg(dev, EPCR, phy ? (EPCR_EPOS | EPCR_ERPRR) : EPCR_ERPRR);
+ sr_write_reg(dev, SR_EPAR, phy ? (reg | EPAR_PHY_ADR) : reg);
+ sr_write_reg(dev, SR_EPCR, phy ? (EPCR_EPOS | EPCR_ERPRR) : EPCR_ERPRR);
ret = wait_phy_eeprom_ready(dev, phy);
if (ret < 0)
goto out_unlock;
- sr_write_reg(dev, EPCR, 0x0);
- ret = sr_read(dev, EPDR, 2, value);
+ sr_write_reg(dev, SR_EPCR, 0x0);
+ ret = sr_read(dev, SR_EPDR, 2, value);
netdev_dbg(dev->net, "read shared %d 0x%02x returned 0x%04x, %d\n",
phy, reg, *value, ret);
mutex_lock(&dev->phy_mutex);
- ret = sr_write(dev, EPDR, 2, &value);
+ ret = sr_write(dev, SR_EPDR, 2, &value);
if (ret < 0)
goto out_unlock;
- sr_write_reg(dev, EPAR, phy ? (reg | EPAR_PHY_ADR) : reg);
- sr_write_reg(dev, EPCR, phy ? (EPCR_WEP | EPCR_EPOS | EPCR_ERPRW) :
+ sr_write_reg(dev, SR_EPAR, phy ? (reg | EPAR_PHY_ADR) : reg);
+ sr_write_reg(dev, SR_EPCR, phy ? (EPCR_WEP | EPCR_EPOS | EPCR_ERPRW) :
(EPCR_WEP | EPCR_ERPRW));
ret = wait_phy_eeprom_ready(dev, phy);
if (ret < 0)
goto out_unlock;
- sr_write_reg(dev, EPCR, 0x0);
+ sr_write_reg(dev, SR_EPCR, 0x0);
out_unlock:
mutex_unlock(&dev->phy_mutex);
if (loc == MII_BMSR) {
u8 value;
- sr_read_reg(dev, NSR, &value);
+ sr_read_reg(dev, SR_NSR, &value);
if (value & NSR_LINKST)
rc = 1;
}
int rc = 0;
/* Get the Link Status directly */
- sr_read_reg(dev, NSR, &value);
+ sr_read_reg(dev, SR_NSR, &value);
if (value & NSR_LINKST)
rc = 1;
}
}
- sr_write_async(dev, MAR, SR_MCAST_SIZE, hashes);
- sr_write_reg_async(dev, RCR, rx_ctl);
+ sr_write_async(dev, SR_MAR, SR_MCAST_SIZE, hashes);
+ sr_write_reg_async(dev, SR_RCR, rx_ctl);
}
static int sr9700_set_mac_address(struct net_device *netdev, void *p)
}
memcpy(netdev->dev_addr, addr->sa_data, netdev->addr_len);
- sr_write_async(dev, PAR, 6, netdev->dev_addr);
+ sr_write_async(dev, SR_PAR, 6, netdev->dev_addr);
return 0;
}
mii->phy_id_mask = 0x1f;
mii->reg_num_mask = 0x1f;
- sr_write_reg(dev, NCR, NCR_RST);
+ sr_write_reg(dev, SR_NCR, NCR_RST);
udelay(20);
/* read MAC
* EEPROM automatically to PAR. In case there is no EEPROM externally,
* a default MAC address is stored in PAR for making chip work properly.
*/
- if (sr_read(dev, PAR, ETH_ALEN, netdev->dev_addr) < 0) {
+ if (sr_read(dev, SR_PAR, ETH_ALEN, netdev->dev_addr) < 0) {
netdev_err(netdev, "Error reading MAC address\n");
ret = -ENODEV;
goto out;
}
/* power up and reset phy */
- sr_write_reg(dev, PRR, PRR_PHY_RST);
+ sr_write_reg(dev, SR_PRR, PRR_PHY_RST);
/* at least 10ms, here 20ms for safe */
mdelay(20);
- sr_write_reg(dev, PRR, 0);
+ sr_write_reg(dev, SR_PRR, 0);
/* at least 1ms, here 2ms for reading right register */
udelay(2 * 1000);
/* sr9700 spec. register table on Linux platform */
/* Network Control Reg */
-#define NCR 0x00
+#define SR_NCR 0x00
#define NCR_RST (1 << 0)
#define NCR_LBK (3 << 1)
#define NCR_FDX (1 << 3)
#define NCR_WAKEEN (1 << 6)
/* Network Status Reg */
-#define NSR 0x01
+#define SR_NSR 0x01
#define NSR_RXRDY (1 << 0)
#define NSR_RXOV (1 << 1)
#define NSR_TX1END (1 << 2)
#define NSR_LINKST (1 << 6)
#define NSR_SPEED (1 << 7)
/* Tx Control Reg */
-#define TCR 0x02
+#define SR_TCR 0x02
#define TCR_CRC_DIS (1 << 1)
#define TCR_PAD_DIS (1 << 2)
#define TCR_LC_CARE (1 << 3)
#define TCR_EXCECM (1 << 5)
#define TCR_LF_EN (1 << 6)
/* Tx Status Reg for Packet Index 1 */
-#define TSR1 0x03
+#define SR_TSR1 0x03
#define TSR1_EC (1 << 2)
#define TSR1_COL (1 << 3)
#define TSR1_LC (1 << 4)
#define TSR1_LOC (1 << 6)
#define TSR1_TLF (1 << 7)
/* Tx Status Reg for Packet Index 2 */
-#define TSR2 0x04
+#define SR_TSR2 0x04
#define TSR2_EC (1 << 2)
#define TSR2_COL (1 << 3)
#define TSR2_LC (1 << 4)
#define TSR2_LOC (1 << 6)
#define TSR2_TLF (1 << 7)
/* Rx Control Reg*/
-#define RCR 0x05
+#define SR_RCR 0x05
#define RCR_RXEN (1 << 0)
#define RCR_PRMSC (1 << 1)
#define RCR_RUNT (1 << 2)
#define RCR_DIS_CRC (1 << 4)
#define RCR_DIS_LONG (1 << 5)
/* Rx Status Reg */
-#define RSR 0x06
+#define SR_RSR 0x06
#define RSR_AE (1 << 2)
#define RSR_MF (1 << 6)
#define RSR_RF (1 << 7)
/* Rx Overflow Counter Reg */
-#define ROCR 0x07
+#define SR_ROCR 0x07
#define ROCR_ROC (0x7F << 0)
#define ROCR_RXFU (1 << 7)
/* Back Pressure Threshold Reg */
-#define BPTR 0x08
+#define SR_BPTR 0x08
#define BPTR_JPT (0x0F << 0)
#define BPTR_BPHW (0x0F << 4)
/* Flow Control Threshold Reg */
-#define FCTR 0x09
+#define SR_FCTR 0x09
#define FCTR_LWOT (0x0F << 0)
#define FCTR_HWOT (0x0F << 4)
/* rx/tx Flow Control Reg */
-#define FCR 0x0A
+#define SR_FCR 0x0A
#define FCR_FLCE (1 << 0)
#define FCR_BKPA (1 << 4)
#define FCR_TXPEN (1 << 5)
#define FCR_TXPF (1 << 6)
#define FCR_TXP0 (1 << 7)
/* Eeprom & Phy Control Reg */
-#define EPCR 0x0B
+#define SR_EPCR 0x0B
#define EPCR_ERRE (1 << 0)
#define EPCR_ERPRW (1 << 1)
#define EPCR_ERPRR (1 << 2)
#define EPCR_EPOS (1 << 3)
#define EPCR_WEP (1 << 4)
/* Eeprom & Phy Address Reg */
-#define EPAR 0x0C
+#define SR_EPAR 0x0C
#define EPAR_EROA (0x3F << 0)
#define EPAR_PHY_ADR_MASK (0x03 << 6)
#define EPAR_PHY_ADR (0x01 << 6)
/* Eeprom & Phy Data Reg */
-#define EPDR 0x0D /* 0x0D ~ 0x0E for Data Reg Low & High */
+#define SR_EPDR 0x0D /* 0x0D ~ 0x0E for Data Reg Low & High */
/* Wakeup Control Reg */
-#define WCR 0x0F
+#define SR_WCR 0x0F
#define WCR_MAGICST (1 << 0)
#define WCR_LINKST (1 << 2)
#define WCR_MAGICEN (1 << 3)
#define WCR_LINKEN (1 << 5)
/* Physical Address Reg */
-#define PAR 0x10 /* 0x10 ~ 0x15 6 bytes for PAR */
+#define SR_PAR 0x10 /* 0x10 ~ 0x15 6 bytes for PAR */
/* Multicast Address Reg */
-#define MAR 0x16 /* 0x16 ~ 0x1D 8 bytes for MAR */
+#define SR_MAR 0x16 /* 0x16 ~ 0x1D 8 bytes for MAR */
/* 0x1e unused */
/* Phy Reset Reg */
-#define PRR 0x1F
+#define SR_PRR 0x1F
#define PRR_PHY_RST (1 << 0)
/* Tx sdram Write Pointer Address Low */
-#define TWPAL 0x20
+#define SR_TWPAL 0x20
/* Tx sdram Write Pointer Address High */
-#define TWPAH 0x21
+#define SR_TWPAH 0x21
/* Tx sdram Read Pointer Address Low */
-#define TRPAL 0x22
+#define SR_TRPAL 0x22
/* Tx sdram Read Pointer Address High */
-#define TRPAH 0x23
+#define SR_TRPAH 0x23
/* Rx sdram Write Pointer Address Low */
-#define RWPAL 0x24
+#define SR_RWPAL 0x24
/* Rx sdram Write Pointer Address High */
-#define RWPAH 0x25
+#define SR_RWPAH 0x25
/* Rx sdram Read Pointer Address Low */
-#define RRPAL 0x26
+#define SR_RRPAL 0x26
/* Rx sdram Read Pointer Address High */
-#define RRPAH 0x27
+#define SR_RRPAH 0x27
/* Vendor ID register */
-#define VID 0x28 /* 0x28 ~ 0x29 2 bytes for VID */
+#define SR_VID 0x28 /* 0x28 ~ 0x29 2 bytes for VID */
/* Product ID register */
-#define PID 0x2A /* 0x2A ~ 0x2B 2 bytes for PID */
+#define SR_PID 0x2A /* 0x2A ~ 0x2B 2 bytes for PID */
/* CHIP Revision register */
-#define CHIPR 0x2C
+#define SR_CHIPR 0x2C
/* 0x2D --> 0xEF unused */
/* USB Device Address */
-#define USBDA 0xF0
+#define SR_USBDA 0xF0
#define USBDA_USBFA (0x7F << 0)
/* RX packet Counter Reg */
-#define RXC 0xF1
+#define SR_RXC 0xF1
/* Tx packet Counter & USB Status Reg */
-#define TXC_USBS 0xF2
+#define SR_TXC_USBS 0xF2
#define TXC_USBS_TXC0 (1 << 0)
#define TXC_USBS_TXC1 (1 << 1)
#define TXC_USBS_TXC2 (1 << 2)
#define TXC_USBS_SUSFLAG (1 << 6)
#define TXC_USBS_RXFAULT (1 << 7)
/* USB Control register */
-#define USBC 0xF4
+#define SR_USBC 0xF4
#define USBC_EP3NAK (1 << 4)
#define USBC_EP3ACK (1 << 5)
skb_shinfo(skb)->gso_type = SKB_GSO_TCPV4;
break;
case VIRTIO_NET_HDR_GSO_UDP:
- {
- static bool warned;
-
- if (!warned) {
- warned = true;
- netdev_warn(dev,
- "host using disabled UFO feature; please fix it\n");
- }
skb_shinfo(skb)->gso_type = SKB_GSO_UDP;
break;
- }
case VIRTIO_NET_HDR_GSO_TCPV6:
skb_shinfo(skb)->gso_type = SKB_GSO_TCPV6;
break;
hdr->hdr.gso_type = VIRTIO_NET_HDR_GSO_TCPV4;
else if (skb_shinfo(skb)->gso_type & SKB_GSO_TCPV6)
hdr->hdr.gso_type = VIRTIO_NET_HDR_GSO_TCPV6;
+ else if (skb_shinfo(skb)->gso_type & SKB_GSO_UDP)
+ hdr->hdr.gso_type = VIRTIO_NET_HDR_GSO_UDP;
else
BUG();
if (skb_shinfo(skb)->gso_type & SKB_GSO_TCP_ECN)
dev->features |= NETIF_F_HW_CSUM|NETIF_F_SG|NETIF_F_FRAGLIST;
if (virtio_has_feature(vdev, VIRTIO_NET_F_GSO)) {
- dev->hw_features |= NETIF_F_TSO
+ dev->hw_features |= NETIF_F_TSO | NETIF_F_UFO
| NETIF_F_TSO_ECN | NETIF_F_TSO6;
}
/* Individual feature bits: what can host handle? */
dev->hw_features |= NETIF_F_TSO6;
if (virtio_has_feature(vdev, VIRTIO_NET_F_HOST_ECN))
dev->hw_features |= NETIF_F_TSO_ECN;
+ if (virtio_has_feature(vdev, VIRTIO_NET_F_HOST_UFO))
+ dev->hw_features |= NETIF_F_UFO;
if (gso)
- dev->features |= dev->hw_features & NETIF_F_ALL_TSO;
+ dev->features |= dev->hw_features & (NETIF_F_ALL_TSO|NETIF_F_UFO);
/* (!csum && gso) case will be fixed by register_netdev() */
}
if (virtio_has_feature(vdev, VIRTIO_NET_F_GUEST_CSUM))
/* If we can receive ANY GSO packets, we must allocate large ones. */
if (virtio_has_feature(vdev, VIRTIO_NET_F_GUEST_TSO4) ||
virtio_has_feature(vdev, VIRTIO_NET_F_GUEST_TSO6) ||
- virtio_has_feature(vdev, VIRTIO_NET_F_GUEST_ECN))
+ virtio_has_feature(vdev, VIRTIO_NET_F_GUEST_ECN) ||
+ virtio_has_feature(vdev, VIRTIO_NET_F_GUEST_UFO))
vi->big_packets = true;
if (virtio_has_feature(vdev, VIRTIO_NET_F_MRG_RXBUF))
static unsigned int features[] = {
VIRTIO_NET_F_CSUM, VIRTIO_NET_F_GUEST_CSUM,
VIRTIO_NET_F_GSO, VIRTIO_NET_F_MAC,
- VIRTIO_NET_F_HOST_TSO4, VIRTIO_NET_F_HOST_TSO6,
+ VIRTIO_NET_F_HOST_TSO4, VIRTIO_NET_F_HOST_UFO, VIRTIO_NET_F_HOST_TSO6,
VIRTIO_NET_F_HOST_ECN, VIRTIO_NET_F_GUEST_TSO4, VIRTIO_NET_F_GUEST_TSO6,
- VIRTIO_NET_F_GUEST_ECN,
+ VIRTIO_NET_F_GUEST_ECN, VIRTIO_NET_F_GUEST_UFO,
VIRTIO_NET_F_MRG_RXBUF, VIRTIO_NET_F_STATUS, VIRTIO_NET_F_CTRL_VQ,
VIRTIO_NET_F_CTRL_RX, VIRTIO_NET_F_CTRL_VLAN,
VIRTIO_NET_F_GUEST_ANNOUNCE, VIRTIO_NET_F_MQ,
dev_put(vxlan->dev);
}
-static int vxlan_newlink(struct net *net, struct net_device *dev,
+static int vxlan_newlink(struct net *src_net, struct net_device *dev,
struct nlattr *tb[], struct nlattr *data[])
{
- struct vxlan_net *vn = net_generic(net, vxlan_net_id);
+ struct vxlan_net *vn = net_generic(src_net, vxlan_net_id);
struct vxlan_dev *vxlan = netdev_priv(dev);
struct vxlan_rdst *dst = &vxlan->default_dst;
__u32 vni;
if (!data[IFLA_VXLAN_ID])
return -EINVAL;
- vxlan->net = dev_net(dev);
+ vxlan->net = src_net;
vni = nla_get_u32(data[IFLA_VXLAN_ID]);
dst->remote_vni = vni;
if (data[IFLA_VXLAN_LINK] &&
(dst->remote_ifindex = nla_get_u32(data[IFLA_VXLAN_LINK]))) {
struct net_device *lowerdev
- = __dev_get_by_index(net, dst->remote_ifindex);
+ = __dev_get_by_index(src_net, dst->remote_ifindex);
if (!lowerdev) {
pr_info("ifindex %d does not exist\n", dst->remote_ifindex);
nla_get_u8(data[IFLA_VXLAN_UDP_ZERO_CSUM6_RX]))
vxlan->flags |= VXLAN_F_UDP_ZERO_CSUM6_RX;
- if (vxlan_find_vni(net, vni, use_ipv6 ? AF_INET6 : AF_INET,
+ if (vxlan_find_vni(src_net, vni, use_ipv6 ? AF_INET6 : AF_INET,
vxlan->dst_port)) {
pr_info("duplicate VNI %u\n", vni);
return -EEXIST;
# There is no way to detect a comtrol sv11 - force it modular for now.
config HOSTESS_SV11
tristate "Comtrol Hostess SV-11 support"
- depends on ISA && m && ISA_DMA_API && INET && HDLC
+ depends on ISA && m && ISA_DMA_API && INET && HDLC && VIRT_TO_BUS
help
Driver for Comtrol Hostess SV-11 network card which
operates on low speed synchronous serial links at up to
# The COSA/SRP driver has not been tested as non-modular yet.
config COSA
tristate "COSA/SRP sync serial boards support"
- depends on ISA && m && ISA_DMA_API && HDLC
+ depends on ISA && m && ISA_DMA_API && HDLC && VIRT_TO_BUS
---help---
Driver for COSA and SRP synchronous serial boards.
# There is no way to detect a Sealevel board. Force it modular
config SEALEVEL_4021
tristate "Sealevel Systems 4021 support"
- depends on ISA && m && ISA_DMA_API && INET && HDLC
+ depends on ISA && m && ISA_DMA_API && INET && HDLC && VIRT_TO_BUS
help
This is a driver for the Sealevel Systems ACB 56 serial I/O adapter.
goto err_rx_unbind;
}
queue->task = task;
+ get_task_struct(task);
task = kthread_create(xenvif_dealloc_kthread,
(void *)queue, "%s-dealloc", queue->name);
if (queue->task) {
kthread_stop(queue->task);
+ put_task_struct(queue->task);
queue->task = NULL;
}
*/
if (unlikely(vif->disabled && queue->id == 0)) {
xenvif_carrier_off(vif);
- xenvif_rx_queue_purge(queue);
- continue;
+ break;
}
if (!skb_queue_empty(&queue->rx_queue))
struct msi_msg msg;
struct pcie_port *pp = sys_to_pcie(pdev->bus->sysdata);
+ if (desc->msi_attrib.is_msix)
+ return -EINVAL;
+
irq = assign_irq(1, desc, &pos);
if (irq < 0)
return irq;
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_S3, PCI_DEVICE_ID_S3_868, quirk_s3_64M);
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_S3, PCI_DEVICE_ID_S3_968, quirk_s3_64M);
+static void quirk_io(struct pci_dev *dev, int pos, unsigned size,
+ const char *name)
+{
+ u32 region;
+ struct pci_bus_region bus_region;
+ struct resource *res = dev->resource + pos;
+
+ pci_read_config_dword(dev, PCI_BASE_ADDRESS_0 + (pos << 2), ®ion);
+
+ if (!region)
+ return;
+
+ res->name = pci_name(dev);
+ res->flags = region & ~PCI_BASE_ADDRESS_IO_MASK;
+ res->flags |=
+ (IORESOURCE_IO | IORESOURCE_PCI_FIXED | IORESOURCE_SIZEALIGN);
+ region &= ~(size - 1);
+
+ /* Convert from PCI bus to resource space */
+ bus_region.start = region;
+ bus_region.end = region + size - 1;
+ pcibios_bus_to_resource(dev->bus, res, &bus_region);
+
+ dev_info(&dev->dev, FW_BUG "%s quirk: reg 0x%x: %pR\n",
+ name, PCI_BASE_ADDRESS_0 + (pos << 2), res);
+}
+
/*
* Some CS5536 BIOSes (for example, the Soekris NET5501 board w/ comBIOS
* ver. 1.33 20070103) don't set the correct ISA PCI region header info.
* BAR0 should be 8 bytes; instead, it may be set to something like 8k
* (which conflicts w/ BAR1's memory range).
+ *
+ * CS553x's ISA PCI BARs may also be read-only (ref:
+ * https://bugzilla.kernel.org/show_bug.cgi?id=85991 - Comment #4 forward).
*/
static void quirk_cs5536_vsa(struct pci_dev *dev)
{
+ static char *name = "CS5536 ISA bridge";
+
if (pci_resource_len(dev, 0) != 8) {
- struct resource *res = &dev->resource[0];
- res->end = res->start + 8 - 1;
- dev_info(&dev->dev, "CS5536 ISA bridge bug detected (incorrect header); workaround applied\n");
+ quirk_io(dev, 0, 8, name); /* SMB */
+ quirk_io(dev, 1, 256, name); /* GPIO */
+ quirk_io(dev, 2, 64, name); /* MFGPT */
+ dev_info(&dev->dev, "%s bug detected (incorrect header); workaround applied\n",
+ name);
}
}
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_CS5536_ISA, quirk_cs5536_vsa);
struct device *dev;
struct pinctrl_dev *pctl;
- int nbanks;
+ int nactive_banks;
uint32_t *mux_mask;
int nmux;
int mux;
/* check if it's a valid config */
- if (pin->bank >= info->nbanks) {
+ if (pin->bank >= gpio_banks) {
dev_err(info->dev, "%s: pin conf %d bank_id %d >= nbanks %d\n",
- name, index, pin->bank, info->nbanks);
+ name, index, pin->bank, gpio_banks);
return -EINVAL;
}
+ if (!gpio_chips[pin->bank]) {
+ dev_err(info->dev, "%s: pin conf %d bank_id %d not enabled\n",
+ name, index, pin->bank);
+ return -ENXIO;
+ }
+
if (pin->pin >= MAX_NB_GPIO_PER_BANK) {
dev_err(info->dev, "%s: pin conf %d pin_bank_id %d >= %d\n",
name, index, pin->pin, MAX_NB_GPIO_PER_BANK);
for_each_child_of_node(np, child) {
if (of_device_is_compatible(child, gpio_compat)) {
- info->nbanks++;
+ if (of_device_is_available(child))
+ info->nactive_banks++;
} else {
info->nfunctions++;
info->ngroups += of_get_child_count(child);
}
size /= sizeof(*list);
- if (!size || size % info->nbanks) {
- dev_err(info->dev, "wrong mux mask array should be by %d\n", info->nbanks);
+ if (!size || size % gpio_banks) {
+ dev_err(info->dev, "wrong mux mask array should be by %d\n", gpio_banks);
return -EINVAL;
}
- info->nmux = size / info->nbanks;
+ info->nmux = size / gpio_banks;
info->mux_mask = devm_kzalloc(info->dev, sizeof(u32) * size, GFP_KERNEL);
if (!info->mux_mask) {
of_match_device(at91_pinctrl_of_match, &pdev->dev)->data;
at91_pinctrl_child_count(info, np);
- if (info->nbanks < 1) {
+ if (gpio_banks < 1) {
dev_err(&pdev->dev, "you need to specify at least one gpio-controller\n");
return -EINVAL;
}
dev_dbg(&pdev->dev, "mux-mask\n");
tmp = info->mux_mask;
- for (i = 0; i < info->nbanks; i++) {
+ for (i = 0; i < gpio_banks; i++) {
for (j = 0; j < info->nmux; j++, tmp++) {
dev_dbg(&pdev->dev, "%d:%d\t0x%x\n", i, j, tmp[0]);
}
if (!info->groups)
return -ENOMEM;
- dev_dbg(&pdev->dev, "nbanks = %d\n", info->nbanks);
+ dev_dbg(&pdev->dev, "nbanks = %d\n", gpio_banks);
dev_dbg(&pdev->dev, "nfunctions = %d\n", info->nfunctions);
dev_dbg(&pdev->dev, "ngroups = %d\n", info->ngroups);
{
struct at91_pinctrl *info;
struct pinctrl_pin_desc *pdesc;
- int ret, i, j, k;
+ int ret, i, j, k, ngpio_chips_enabled = 0;
info = devm_kzalloc(&pdev->dev, sizeof(*info), GFP_KERNEL);
if (!info)
* to obtain references to the struct gpio_chip * for them, and we
* need this to proceed.
*/
- for (i = 0; i < info->nbanks; i++) {
- if (!gpio_chips[i]) {
- dev_warn(&pdev->dev, "GPIO chip %d not registered yet\n", i);
- devm_kfree(&pdev->dev, info);
- return -EPROBE_DEFER;
- }
+ for (i = 0; i < gpio_banks; i++)
+ if (gpio_chips[i])
+ ngpio_chips_enabled++;
+
+ if (ngpio_chips_enabled < info->nactive_banks) {
+ dev_warn(&pdev->dev,
+ "All GPIO chips are not registered yet (%d/%d)\n",
+ ngpio_chips_enabled, info->nactive_banks);
+ devm_kfree(&pdev->dev, info);
+ return -EPROBE_DEFER;
}
at91_pinctrl_desc.name = dev_name(&pdev->dev);
- at91_pinctrl_desc.npins = info->nbanks * MAX_NB_GPIO_PER_BANK;
+ at91_pinctrl_desc.npins = gpio_banks * MAX_NB_GPIO_PER_BANK;
at91_pinctrl_desc.pins = pdesc =
devm_kzalloc(&pdev->dev, sizeof(*pdesc) * at91_pinctrl_desc.npins, GFP_KERNEL);
if (!at91_pinctrl_desc.pins)
return -ENOMEM;
- for (i = 0 , k = 0; i < info->nbanks; i++) {
+ for (i = 0, k = 0; i < gpio_banks; i++) {
for (j = 0; j < MAX_NB_GPIO_PER_BANK; j++, k++) {
pdesc->number = k;
pdesc->name = kasprintf(GFP_KERNEL, "pio%c%d", i + 'A', j);
}
/* We will handle a range of GPIO pins */
- for (i = 0; i < info->nbanks; i++)
- pinctrl_add_gpio_range(info->pctl, &gpio_chips[i]->range);
+ for (i = 0; i < gpio_banks; i++)
+ if (gpio_chips[i])
+ pinctrl_add_gpio_range(info->pctl, &gpio_chips[i]->range);
dev_info(&pdev->dev, "initialized AT91 pinctrl driver\n");
static int at91_gpio_of_irq_setup(struct platform_device *pdev,
struct at91_gpio_chip *at91_gpio)
{
+ struct gpio_chip *gpiochip_prev = NULL;
struct at91_gpio_chip *prev = NULL;
struct irq_data *d = irq_get_irq_data(at91_gpio->pioc_virq);
- int ret;
+ int ret, i;
at91_gpio->pioc_hwirq = irqd_to_hwirq(d);
return ret;
}
- /* Setup chained handler */
- if (at91_gpio->pioc_idx)
- prev = gpio_chips[at91_gpio->pioc_idx - 1];
-
/* The top level handler handles one bank of GPIOs, except
* on some SoC it can handle up to three...
* We only set up the handler for the first of the list.
*/
- if (prev && prev->next == at91_gpio)
+ gpiochip_prev = irq_get_handler_data(at91_gpio->pioc_virq);
+ if (!gpiochip_prev) {
+ /* Then register the chain on the parent IRQ */
+ gpiochip_set_chained_irqchip(&at91_gpio->chip,
+ &gpio_irqchip,
+ at91_gpio->pioc_virq,
+ gpio_irq_handler);
return 0;
+ }
- /* Then register the chain on the parent IRQ */
- gpiochip_set_chained_irqchip(&at91_gpio->chip,
- &gpio_irqchip,
- at91_gpio->pioc_virq,
- gpio_irq_handler);
+ prev = container_of(gpiochip_prev, struct at91_gpio_chip, chip);
- return 0;
+ /* we can only have 2 banks before */
+ for (i = 0; i < 2; i++) {
+ if (prev->next) {
+ prev = prev->next;
+ } else {
+ prev->next = at91_gpio;
+ return 0;
+ }
+ }
+
+ return -EINVAL;
}
/* This structure is replicated for each GPIO block allocated at probe time */
.ngpio = MAX_NB_GPIO_PER_BANK,
};
-static void at91_gpio_probe_fixup(void)
-{
- unsigned i;
- struct at91_gpio_chip *at91_gpio, *last = NULL;
-
- for (i = 0; i < gpio_banks; i++) {
- at91_gpio = gpio_chips[i];
-
- /*
- * GPIO controller are grouped on some SoC:
- * PIOC, PIOD and PIOE can share the same IRQ line
- */
- if (last && last->pioc_virq == at91_gpio->pioc_virq)
- last->next = at91_gpio;
- last = at91_gpio;
- }
-}
-
static struct of_device_id at91_gpio_of_match[] = {
{ .compatible = "atmel,at91sam9x5-gpio", .data = &at91sam9x5_ops, },
{ .compatible = "atmel,at91rm9200-gpio", .data = &at91rm9200_ops },
gpio_chips[alias_idx] = at91_chip;
gpio_banks = max(gpio_banks, alias_idx + 1);
- at91_gpio_probe_fixup();
-
ret = at91_gpio_of_irq_setup(pdev, at91_chip);
if (ret)
goto irq_setup_err;
Exynos5420/Exynos5800 SoCs to control various voltages.
It includes support for control of voltage and ramp speed.
+config REGULATOR_MAX77843
+ tristate "Maxim 77843 regulator"
+ depends on MFD_MAX77843
+ help
+ This driver controls a Maxim 77843 regulator.
+ The regulator include two 'SAFEOUT' for USB(Universal Serial Bus)
+ This is suitable for Exynos5433 SoC chips.
+
config REGULATOR_MC13XXX_CORE
tristate
Say y here to support the regulators found on the Freescale MC13892
PMIC.
+config REGULATOR_MT6397
+ tristate "MediaTek MT6397 PMIC"
+ depends on MFD_MT6397
+ help
+ Say y here to select this option to enable the power regulator of
+ MediaTek MT6397 PMIC.
+ This driver supports the control of different power rails of device
+ through regulator interface.
+
config REGULATOR_PALMAS
tristate "TI Palmas PMIC Regulators"
depends on MFD_PALMAS
obj-$(CONFIG_REGULATOR_MAX77686) += max77686.o
obj-$(CONFIG_REGULATOR_MAX77693) += max77693.o
obj-$(CONFIG_REGULATOR_MAX77802) += max77802.o
+obj-$(CONFIG_REGULATOR_MAX77843) += max77843.o
obj-$(CONFIG_REGULATOR_MC13783) += mc13783-regulator.o
obj-$(CONFIG_REGULATOR_MC13892) += mc13892-regulator.o
obj-$(CONFIG_REGULATOR_MC13XXX_CORE) += mc13xxx-regulator-core.o
+obj-$(CONFIG_REGULATOR_MT6397) += mt6397-regulator.o
obj-$(CONFIG_REGULATOR_QCOM_RPM) += qcom_rpm-regulator.o
obj-$(CONFIG_REGULATOR_PALMAS) += palmas-regulator.o
obj-$(CONFIG_REGULATOR_PFUZE100) += pfuze100-regulator.o
#define AXP20X_FREQ_DCDC_MASK 0x0f
-#define AXP20X_DESC_IO(_id, _supply, _min, _max, _step, _vreg, _vmask, _ereg, \
- _emask, _enable_val, _disable_val) \
+#define AXP20X_DESC_IO(_id, _match, _supply, _min, _max, _step, _vreg, _vmask, \
+ _ereg, _emask, _enable_val, _disable_val) \
[AXP20X_##_id] = { \
.name = #_id, \
.supply_name = (_supply), \
+ .of_match = of_match_ptr(_match), \
+ .regulators_node = of_match_ptr("regulators"), \
.type = REGULATOR_VOLTAGE, \
.id = AXP20X_##_id, \
.n_voltages = (((_max) - (_min)) / (_step) + 1), \
.ops = &axp20x_ops, \
}
-#define AXP20X_DESC(_id, _supply, _min, _max, _step, _vreg, _vmask, _ereg, \
- _emask) \
+#define AXP20X_DESC(_id, _match, _supply, _min, _max, _step, _vreg, _vmask, \
+ _ereg, _emask) \
[AXP20X_##_id] = { \
.name = #_id, \
.supply_name = (_supply), \
+ .of_match = of_match_ptr(_match), \
+ .regulators_node = of_match_ptr("regulators"), \
.type = REGULATOR_VOLTAGE, \
.id = AXP20X_##_id, \
.n_voltages = (((_max) - (_min)) / (_step) + 1), \
.ops = &axp20x_ops, \
}
-#define AXP20X_DESC_FIXED(_id, _supply, _volt) \
+#define AXP20X_DESC_FIXED(_id, _match, _supply, _volt) \
[AXP20X_##_id] = { \
.name = #_id, \
.supply_name = (_supply), \
+ .of_match = of_match_ptr(_match), \
+ .regulators_node = of_match_ptr("regulators"), \
.type = REGULATOR_VOLTAGE, \
.id = AXP20X_##_id, \
.n_voltages = 1, \
.ops = &axp20x_ops_fixed \
}
-#define AXP20X_DESC_TABLE(_id, _supply, _table, _vreg, _vmask, _ereg, _emask) \
+#define AXP20X_DESC_TABLE(_id, _match, _supply, _table, _vreg, _vmask, _ereg, \
+ _emask) \
[AXP20X_##_id] = { \
.name = #_id, \
.supply_name = (_supply), \
+ .of_match = of_match_ptr(_match), \
+ .regulators_node = of_match_ptr("regulators"), \
.type = REGULATOR_VOLTAGE, \
.id = AXP20X_##_id, \
.n_voltages = ARRAY_SIZE(_table), \
};
static const struct regulator_desc axp20x_regulators[] = {
- AXP20X_DESC(DCDC2, "vin2", 700, 2275, 25, AXP20X_DCDC2_V_OUT, 0x3f,
- AXP20X_PWR_OUT_CTRL, 0x10),
- AXP20X_DESC(DCDC3, "vin3", 700, 3500, 25, AXP20X_DCDC3_V_OUT, 0x7f,
- AXP20X_PWR_OUT_CTRL, 0x02),
- AXP20X_DESC_FIXED(LDO1, "acin", 1300),
- AXP20X_DESC(LDO2, "ldo24in", 1800, 3300, 100, AXP20X_LDO24_V_OUT, 0xf0,
- AXP20X_PWR_OUT_CTRL, 0x04),
- AXP20X_DESC(LDO3, "ldo3in", 700, 3500, 25, AXP20X_LDO3_V_OUT, 0x7f,
- AXP20X_PWR_OUT_CTRL, 0x40),
- AXP20X_DESC_TABLE(LDO4, "ldo24in", axp20x_ldo4_data, AXP20X_LDO24_V_OUT, 0x0f,
- AXP20X_PWR_OUT_CTRL, 0x08),
- AXP20X_DESC_IO(LDO5, "ldo5in", 1800, 3300, 100, AXP20X_LDO5_V_OUT, 0xf0,
- AXP20X_GPIO0_CTRL, 0x07, AXP20X_IO_ENABLED,
- AXP20X_IO_DISABLED),
-};
-
-#define AXP_MATCH(_name, _id) \
- [AXP20X_##_id] = { \
- .name = #_name, \
- .driver_data = (void *) &axp20x_regulators[AXP20X_##_id], \
- }
-
-static struct of_regulator_match axp20x_matches[] = {
- AXP_MATCH(dcdc2, DCDC2),
- AXP_MATCH(dcdc3, DCDC3),
- AXP_MATCH(ldo1, LDO1),
- AXP_MATCH(ldo2, LDO2),
- AXP_MATCH(ldo3, LDO3),
- AXP_MATCH(ldo4, LDO4),
- AXP_MATCH(ldo5, LDO5),
+ AXP20X_DESC(DCDC2, "dcdc2", "vin2", 700, 2275, 25, AXP20X_DCDC2_V_OUT,
+ 0x3f, AXP20X_PWR_OUT_CTRL, 0x10),
+ AXP20X_DESC(DCDC3, "dcdc3", "vin3", 700, 3500, 25, AXP20X_DCDC3_V_OUT,
+ 0x7f, AXP20X_PWR_OUT_CTRL, 0x02),
+ AXP20X_DESC_FIXED(LDO1, "ldo1", "acin", 1300),
+ AXP20X_DESC(LDO2, "ldo2", "ldo24in", 1800, 3300, 100,
+ AXP20X_LDO24_V_OUT, 0xf0, AXP20X_PWR_OUT_CTRL, 0x04),
+ AXP20X_DESC(LDO3, "ldo3", "ldo3in", 700, 3500, 25, AXP20X_LDO3_V_OUT,
+ 0x7f, AXP20X_PWR_OUT_CTRL, 0x40),
+ AXP20X_DESC_TABLE(LDO4, "ldo4", "ldo24in", axp20x_ldo4_data,
+ AXP20X_LDO24_V_OUT, 0x0f, AXP20X_PWR_OUT_CTRL, 0x08),
+ AXP20X_DESC_IO(LDO5, "ldo5", "ldo5in", 1800, 3300, 100,
+ AXP20X_LDO5_V_OUT, 0xf0, AXP20X_GPIO0_CTRL, 0x07,
+ AXP20X_IO_ENABLED, AXP20X_IO_DISABLED),
};
static int axp20x_set_dcdc_freq(struct platform_device *pdev, u32 dcdcfreq)
if (!regulators) {
dev_warn(&pdev->dev, "regulators node not found\n");
} else {
- ret = of_regulator_match(&pdev->dev, regulators, axp20x_matches,
- ARRAY_SIZE(axp20x_matches));
- if (ret < 0) {
- dev_err(&pdev->dev, "Error parsing regulator init data: %d\n", ret);
- return ret;
- }
-
dcdcfreq = 1500;
of_property_read_u32(regulators, "x-powers,dcdc-freq", &dcdcfreq);
ret = axp20x_set_dcdc_freq(pdev, dcdcfreq);
{
struct regulator_dev *rdev;
struct axp20x_dev *axp20x = dev_get_drvdata(pdev->dev.parent);
- struct regulator_config config = { };
- struct regulator_init_data *init_data;
+ struct regulator_config config = {
+ .dev = pdev->dev.parent,
+ .regmap = axp20x->regmap,
+ };
int ret, i;
u32 workmode;
- ret = axp20x_regulator_parse_dt(pdev);
- if (ret)
- return ret;
+ /* This only sets the dcdc freq. Ignore any errors */
+ axp20x_regulator_parse_dt(pdev);
for (i = 0; i < AXP20X_REG_ID_MAX; i++) {
- init_data = axp20x_matches[i].init_data;
-
- config.dev = pdev->dev.parent;
- config.init_data = init_data;
- config.regmap = axp20x->regmap;
- config.of_node = axp20x_matches[i].of_node;
-
rdev = devm_regulator_register(&pdev->dev, &axp20x_regulators[i],
&config);
if (IS_ERR(rdev)) {
return PTR_ERR(rdev);
}
- ret = of_property_read_u32(axp20x_matches[i].of_node, "x-powers,dcdc-workmode",
+ ret = of_property_read_u32(rdev->dev.of_node,
+ "x-powers,dcdc-workmode",
&workmode);
if (!ret) {
if (axp20x_set_dcdc_workmode(rdev, i, workmode))
static DEVICE_ATTR(bypass, 0444,
regulator_bypass_show, NULL);
-/*
- * These are the only attributes are present for all regulators.
- * Other attributes are a function of regulator functionality.
- */
-static struct attribute *regulator_dev_attrs[] = {
- &dev_attr_name.attr,
- &dev_attr_num_users.attr,
- &dev_attr_type.attr,
- NULL,
-};
-ATTRIBUTE_GROUPS(regulator_dev);
-
-static void regulator_dev_release(struct device *dev)
-{
- struct regulator_dev *rdev = dev_get_drvdata(dev);
- kfree(rdev);
-}
-
-static struct class regulator_class = {
- .name = "regulator",
- .dev_release = regulator_dev_release,
- .dev_groups = regulator_dev_groups,
-};
-
/* Calculate the new optimum regulator operating mode based on the new total
* consumer load. All locks held by caller */
-static void drms_uA_update(struct regulator_dev *rdev)
+static int drms_uA_update(struct regulator_dev *rdev)
{
struct regulator *sibling;
int current_uA = 0, output_uV, input_uV, err;
unsigned int mode;
+ /*
+ * first check to see if we can set modes at all, otherwise just
+ * tell the consumer everything is OK.
+ */
err = regulator_check_drms(rdev);
- if (err < 0 || !rdev->desc->ops->get_optimum_mode ||
- (!rdev->desc->ops->get_voltage &&
- !rdev->desc->ops->get_voltage_sel) ||
- !rdev->desc->ops->set_mode)
- return;
+ if (err < 0)
+ return 0;
+
+ if (!rdev->desc->ops->get_optimum_mode)
+ return 0;
+
+ if (!rdev->desc->ops->set_mode)
+ return -EINVAL;
/* get output voltage */
output_uV = _regulator_get_voltage(rdev);
- if (output_uV <= 0)
- return;
+ if (output_uV <= 0) {
+ rdev_err(rdev, "invalid output voltage found\n");
+ return -EINVAL;
+ }
/* get input voltage */
input_uV = 0;
input_uV = regulator_get_voltage(rdev->supply);
if (input_uV <= 0)
input_uV = rdev->constraints->input_uV;
- if (input_uV <= 0)
- return;
+ if (input_uV <= 0) {
+ rdev_err(rdev, "invalid input voltage found\n");
+ return -EINVAL;
+ }
/* calc total requested load */
list_for_each_entry(sibling, &rdev->consumer_list, list)
/* check the new mode is allowed */
err = regulator_mode_constrain(rdev, &mode);
- if (err == 0)
- rdev->desc->ops->set_mode(rdev, mode);
+ if (err < 0) {
+ rdev_err(rdev, "failed to get optimum mode @ %d uA %d -> %d uV\n",
+ current_uA, input_uV, output_uV);
+ return err;
+ }
+
+ err = rdev->desc->ops->set_mode(rdev, mode);
+ if (err < 0)
+ rdev_err(rdev, "failed to set optimum mode %x\n", mode);
+
+ return err;
}
static int suspend_set_state(struct regulator_dev *rdev,
int regulator_set_optimum_mode(struct regulator *regulator, int uA_load)
{
struct regulator_dev *rdev = regulator->rdev;
- struct regulator *consumer;
- int ret, output_uV, input_uV = 0, total_uA_load = 0;
- unsigned int mode;
-
- if (rdev->supply)
- input_uV = regulator_get_voltage(rdev->supply);
+ int ret;
mutex_lock(&rdev->mutex);
-
- /*
- * first check to see if we can set modes at all, otherwise just
- * tell the consumer everything is OK.
- */
regulator->uA_load = uA_load;
- ret = regulator_check_drms(rdev);
- if (ret < 0) {
- ret = 0;
- goto out;
- }
-
- if (!rdev->desc->ops->get_optimum_mode)
- goto out;
-
- /*
- * we can actually do this so any errors are indicators of
- * potential real failure.
- */
- ret = -EINVAL;
-
- if (!rdev->desc->ops->set_mode)
- goto out;
-
- /* get output voltage */
- output_uV = _regulator_get_voltage(rdev);
- if (output_uV <= 0) {
- rdev_err(rdev, "invalid output voltage found\n");
- goto out;
- }
-
- /* No supply? Use constraint voltage */
- if (input_uV <= 0)
- input_uV = rdev->constraints->input_uV;
- if (input_uV <= 0) {
- rdev_err(rdev, "invalid input voltage found\n");
- goto out;
- }
-
- /* calc total requested load for this regulator */
- list_for_each_entry(consumer, &rdev->consumer_list, list)
- total_uA_load += consumer->uA_load;
-
- mode = rdev->desc->ops->get_optimum_mode(rdev,
- input_uV, output_uV,
- total_uA_load);
- ret = regulator_mode_constrain(rdev, &mode);
- if (ret < 0) {
- rdev_err(rdev, "failed to get optimum mode @ %d uA %d -> %d uV\n",
- total_uA_load, input_uV, output_uV);
- goto out;
- }
-
- ret = rdev->desc->ops->set_mode(rdev, mode);
- if (ret < 0) {
- rdev_err(rdev, "failed to set optimum mode %x\n", mode);
- goto out;
- }
- ret = mode;
-out:
+ ret = drms_uA_update(rdev);
mutex_unlock(&rdev->mutex);
+
return ret;
}
EXPORT_SYMBOL_GPL(regulator_set_optimum_mode);
}
EXPORT_SYMBOL_GPL(regulator_mode_to_status);
+static struct attribute *regulator_dev_attrs[] = {
+ &dev_attr_name.attr,
+ &dev_attr_num_users.attr,
+ &dev_attr_type.attr,
+ &dev_attr_microvolts.attr,
+ &dev_attr_microamps.attr,
+ &dev_attr_opmode.attr,
+ &dev_attr_state.attr,
+ &dev_attr_status.attr,
+ &dev_attr_bypass.attr,
+ &dev_attr_requested_microamps.attr,
+ &dev_attr_min_microvolts.attr,
+ &dev_attr_max_microvolts.attr,
+ &dev_attr_min_microamps.attr,
+ &dev_attr_max_microamps.attr,
+ &dev_attr_suspend_standby_state.attr,
+ &dev_attr_suspend_mem_state.attr,
+ &dev_attr_suspend_disk_state.attr,
+ &dev_attr_suspend_standby_microvolts.attr,
+ &dev_attr_suspend_mem_microvolts.attr,
+ &dev_attr_suspend_disk_microvolts.attr,
+ &dev_attr_suspend_standby_mode.attr,
+ &dev_attr_suspend_mem_mode.attr,
+ &dev_attr_suspend_disk_mode.attr,
+ NULL
+};
+
/*
* To avoid cluttering sysfs (and memory) with useless state, only
* create attributes that can be meaningfully displayed.
*/
-static int add_regulator_attributes(struct regulator_dev *rdev)
+static umode_t regulator_attr_is_visible(struct kobject *kobj,
+ struct attribute *attr, int idx)
{
- struct device *dev = &rdev->dev;
+ struct device *dev = kobj_to_dev(kobj);
+ struct regulator_dev *rdev = container_of(dev, struct regulator_dev, dev);
const struct regulator_ops *ops = rdev->desc->ops;
- int status = 0;
+ umode_t mode = attr->mode;
+
+ /* these three are always present */
+ if (attr == &dev_attr_name.attr ||
+ attr == &dev_attr_num_users.attr ||
+ attr == &dev_attr_type.attr)
+ return mode;
/* some attributes need specific methods to be displayed */
- if ((ops->get_voltage && ops->get_voltage(rdev) >= 0) ||
- (ops->get_voltage_sel && ops->get_voltage_sel(rdev) >= 0) ||
- (ops->list_voltage && ops->list_voltage(rdev, 0) >= 0) ||
- (rdev->desc->fixed_uV && (rdev->desc->n_voltages == 1))) {
- status = device_create_file(dev, &dev_attr_microvolts);
- if (status < 0)
- return status;
- }
- if (ops->get_current_limit) {
- status = device_create_file(dev, &dev_attr_microamps);
- if (status < 0)
- return status;
- }
- if (ops->get_mode) {
- status = device_create_file(dev, &dev_attr_opmode);
- if (status < 0)
- return status;
- }
- if (rdev->ena_pin || ops->is_enabled) {
- status = device_create_file(dev, &dev_attr_state);
- if (status < 0)
- return status;
- }
- if (ops->get_status) {
- status = device_create_file(dev, &dev_attr_status);
- if (status < 0)
- return status;
- }
- if (ops->get_bypass) {
- status = device_create_file(dev, &dev_attr_bypass);
- if (status < 0)
- return status;
+ if (attr == &dev_attr_microvolts.attr) {
+ if ((ops->get_voltage && ops->get_voltage(rdev) >= 0) ||
+ (ops->get_voltage_sel && ops->get_voltage_sel(rdev) >= 0) ||
+ (ops->list_voltage && ops->list_voltage(rdev, 0) >= 0) ||
+ (rdev->desc->fixed_uV && rdev->desc->n_voltages == 1))
+ return mode;
+ return 0;
}
+ if (attr == &dev_attr_microamps.attr)
+ return ops->get_current_limit ? mode : 0;
+
+ if (attr == &dev_attr_opmode.attr)
+ return ops->get_mode ? mode : 0;
+
+ if (attr == &dev_attr_state.attr)
+ return (rdev->ena_pin || ops->is_enabled) ? mode : 0;
+
+ if (attr == &dev_attr_status.attr)
+ return ops->get_status ? mode : 0;
+
+ if (attr == &dev_attr_bypass.attr)
+ return ops->get_bypass ? mode : 0;
+
/* some attributes are type-specific */
- if (rdev->desc->type == REGULATOR_CURRENT) {
- status = device_create_file(dev, &dev_attr_requested_microamps);
- if (status < 0)
- return status;
- }
+ if (attr == &dev_attr_requested_microamps.attr)
+ return rdev->desc->type == REGULATOR_CURRENT ? mode : 0;
/* all the other attributes exist to support constraints;
* don't show them if there are no constraints, or if the
* relevant supporting methods are missing.
*/
if (!rdev->constraints)
- return status;
+ return 0;
/* constraints need specific supporting methods */
- if (ops->set_voltage || ops->set_voltage_sel) {
- status = device_create_file(dev, &dev_attr_min_microvolts);
- if (status < 0)
- return status;
- status = device_create_file(dev, &dev_attr_max_microvolts);
- if (status < 0)
- return status;
- }
- if (ops->set_current_limit) {
- status = device_create_file(dev, &dev_attr_min_microamps);
- if (status < 0)
- return status;
- status = device_create_file(dev, &dev_attr_max_microamps);
- if (status < 0)
- return status;
- }
-
- status = device_create_file(dev, &dev_attr_suspend_standby_state);
- if (status < 0)
- return status;
- status = device_create_file(dev, &dev_attr_suspend_mem_state);
- if (status < 0)
- return status;
- status = device_create_file(dev, &dev_attr_suspend_disk_state);
- if (status < 0)
- return status;
+ if (attr == &dev_attr_min_microvolts.attr ||
+ attr == &dev_attr_max_microvolts.attr)
+ return (ops->set_voltage || ops->set_voltage_sel) ? mode : 0;
+
+ if (attr == &dev_attr_min_microamps.attr ||
+ attr == &dev_attr_max_microamps.attr)
+ return ops->set_current_limit ? mode : 0;
- if (ops->set_suspend_voltage) {
- status = device_create_file(dev,
- &dev_attr_suspend_standby_microvolts);
- if (status < 0)
- return status;
- status = device_create_file(dev,
- &dev_attr_suspend_mem_microvolts);
- if (status < 0)
- return status;
- status = device_create_file(dev,
- &dev_attr_suspend_disk_microvolts);
- if (status < 0)
- return status;
- }
-
- if (ops->set_suspend_mode) {
- status = device_create_file(dev,
- &dev_attr_suspend_standby_mode);
- if (status < 0)
- return status;
- status = device_create_file(dev,
- &dev_attr_suspend_mem_mode);
- if (status < 0)
- return status;
- status = device_create_file(dev,
- &dev_attr_suspend_disk_mode);
- if (status < 0)
- return status;
- }
-
- return status;
+ if (attr == &dev_attr_suspend_standby_state.attr ||
+ attr == &dev_attr_suspend_mem_state.attr ||
+ attr == &dev_attr_suspend_disk_state.attr)
+ return mode;
+
+ if (attr == &dev_attr_suspend_standby_microvolts.attr ||
+ attr == &dev_attr_suspend_mem_microvolts.attr ||
+ attr == &dev_attr_suspend_disk_microvolts.attr)
+ return ops->set_suspend_voltage ? mode : 0;
+
+ if (attr == &dev_attr_suspend_standby_mode.attr ||
+ attr == &dev_attr_suspend_mem_mode.attr ||
+ attr == &dev_attr_suspend_disk_mode.attr)
+ return ops->set_suspend_mode ? mode : 0;
+
+ return mode;
}
+static const struct attribute_group regulator_dev_group = {
+ .attrs = regulator_dev_attrs,
+ .is_visible = regulator_attr_is_visible,
+};
+
+static const struct attribute_group *regulator_dev_groups[] = {
+ ®ulator_dev_group,
+ NULL
+};
+
+static void regulator_dev_release(struct device *dev)
+{
+ struct regulator_dev *rdev = dev_get_drvdata(dev);
+ kfree(rdev);
+}
+
+static struct class regulator_class = {
+ .name = "regulator",
+ .dev_release = regulator_dev_release,
+ .dev_groups = regulator_dev_groups,
+};
+
static void rdev_init_debugfs(struct regulator_dev *rdev)
{
rdev->debugfs = debugfs_create_dir(rdev_get_name(rdev), debugfs_root);
/**
* regulator_register - register regulator
* @regulator_desc: regulator to register
- * @config: runtime configuration for regulator
+ * @cfg: runtime configuration for regulator
*
* Called by regulator drivers to register a regulator.
* Returns a valid pointer to struct regulator_dev on success
*/
struct regulator_dev *
regulator_register(const struct regulator_desc *regulator_desc,
- const struct regulator_config *config)
+ const struct regulator_config *cfg)
{
const struct regulation_constraints *constraints = NULL;
const struct regulator_init_data *init_data;
- static atomic_t regulator_no = ATOMIC_INIT(0);
+ struct regulator_config *config = NULL;
+ static atomic_t regulator_no = ATOMIC_INIT(-1);
struct regulator_dev *rdev;
struct device *dev;
int ret, i;
const char *supply = NULL;
- if (regulator_desc == NULL || config == NULL)
+ if (regulator_desc == NULL || cfg == NULL)
return ERR_PTR(-EINVAL);
- dev = config->dev;
+ dev = cfg->dev;
WARN_ON(!dev);
if (regulator_desc->name == NULL || regulator_desc->ops == NULL)
if (rdev == NULL)
return ERR_PTR(-ENOMEM);
- init_data = regulator_of_get_init_data(dev, regulator_desc,
+ /*
+ * Duplicate the config so the driver could override it after
+ * parsing init data.
+ */
+ config = kmemdup(cfg, sizeof(*cfg), GFP_KERNEL);
+ if (config == NULL) {
+ kfree(rdev);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ init_data = regulator_of_get_init_data(dev, regulator_desc, config,
&rdev->dev.of_node);
if (!init_data) {
init_data = config->init_data;
/* register with sysfs */
rdev->dev.class = ®ulator_class;
rdev->dev.parent = dev;
- dev_set_name(&rdev->dev, "regulator.%d",
- atomic_inc_return(®ulator_no) - 1);
+ dev_set_name(&rdev->dev, "regulator.%lu",
+ (unsigned long) atomic_inc_return(®ulator_no));
ret = device_register(&rdev->dev);
if (ret != 0) {
put_device(&rdev->dev);
if (ret < 0)
goto scrub;
- /* add attributes supported by this regulator */
- ret = add_regulator_attributes(rdev);
- if (ret < 0)
- goto scrub;
-
if (init_data && init_data->supply_regulator)
supply = init_data->supply_regulator;
else if (regulator_desc->supply_name)
rdev_init_debugfs(rdev);
out:
mutex_unlock(®ulator_list_mutex);
+ kfree(config);
return rdev;
unset_supplies:
#include <linux/regmap.h>
#include <linux/irq.h>
#include <linux/interrupt.h>
+#include <linux/of_gpio.h>
#include <linux/regulator/of_regulator.h>
#include <linux/regulator/da9211.h>
#include "da9211-regulator.h"
continue;
pdata->init_data[n] = da9211_matches[i].init_data;
-
+ pdata->reg_node[n] = da9211_matches[i].of_node;
+ pdata->gpio_ren[n] =
+ of_get_named_gpio(da9211_matches[i].of_node,
+ "enable-gpios", 0);
n++;
}
config.dev = chip->dev;
config.driver_data = chip;
config.regmap = chip->regmap;
- config.of_node = chip->dev->of_node;
+ config.of_node = chip->pdata->reg_node[i];
+
+ if (gpio_is_valid(chip->pdata->gpio_ren[i])) {
+ config.ena_gpio = chip->pdata->gpio_ren[i];
+ config.ena_gpio_initialized = true;
+ } else {
+ config.ena_gpio = -EINVAL;
+ config.ena_gpio_initialized = false;
+ }
chip->rdev[i] = devm_regulator_register(chip->dev,
&da9211_regulators[i], &config);
return REGULATOR_MODE_NORMAL;
}
-static int slew_rates[] = {
+static const int slew_rates[] = {
64000,
32000,
16000,
return PTR_ERR_OR_ZERO(di->rdev);
}
-static struct regmap_config fan53555_regmap_config = {
+static const struct regmap_config fan53555_regmap_config = {
.reg_bits = 8,
.val_bits = 8,
};
#ifdef CONFIG_OF
struct regulator_init_data *regulator_of_get_init_data(struct device *dev,
const struct regulator_desc *desc,
+ struct regulator_config *config,
struct device_node **node);
#else
static inline struct regulator_init_data *
regulator_of_get_init_data(struct device *dev,
const struct regulator_desc *desc,
+ struct regulator_config *config,
struct device_node **node)
{
return NULL;
#ifdef CONFIG_OF
static const struct of_device_id isl9305_dt_ids[] = {
- { .compatible = "isl,isl9305" },
- { .compatible = "isl,isl9305h" },
+ { .compatible = "isl,isl9305" }, /* for backward compat., don't use */
+ { .compatible = "isil,isl9305" },
+ { .compatible = "isl,isl9305h" }, /* for backward compat., don't use */
+ { .compatible = "isil,isl9305h" },
{},
};
#endif
struct device *dev;
enum lp872x_id chipid;
struct lp872x_platform_data *pdata;
- struct regulator_dev **regulators;
int num_regulators;
enum lp872x_dvs_state dvs_pin;
int dvs_gpio;
dev_err(lp->dev, "regulator register err");
return PTR_ERR(rdev);
}
-
- *(lp->regulators + i) = rdev;
}
return 0;
static int lp872x_probe(struct i2c_client *cl, const struct i2c_device_id *id)
{
struct lp872x *lp;
- int ret, size, num_regulators;
+ int ret;
const int lp872x_num_regulators[] = {
[LP8720] = LP8720_NUM_REGULATORS,
[LP8725] = LP8725_NUM_REGULATORS,
lp = devm_kzalloc(&cl->dev, sizeof(struct lp872x), GFP_KERNEL);
if (!lp)
- goto err_mem;
-
- num_regulators = lp872x_num_regulators[id->driver_data];
- size = sizeof(struct regulator_dev *) * num_regulators;
+ return -ENOMEM;
- lp->regulators = devm_kzalloc(&cl->dev, size, GFP_KERNEL);
- if (!lp->regulators)
- goto err_mem;
+ lp->num_regulators = lp872x_num_regulators[id->driver_data];
lp->regmap = devm_regmap_init_i2c(cl, &lp872x_regmap_config);
if (IS_ERR(lp->regmap)) {
ret = PTR_ERR(lp->regmap);
dev_err(&cl->dev, "regmap init i2c err: %d\n", ret);
- goto err_dev;
+ return ret;
}
lp->dev = &cl->dev;
lp->pdata = dev_get_platdata(&cl->dev);
lp->chipid = id->driver_data;
- lp->num_regulators = num_regulators;
i2c_set_clientdata(cl, lp);
ret = lp872x_config(lp);
if (ret)
- goto err_dev;
+ return ret;
return lp872x_regulator_register(lp);
-
-err_mem:
- return -ENOMEM;
-err_dev:
- return ret;
}
static const struct of_device_id lp872x_dt_ids[] = {
static const struct regulator_desc max14577_supported_regulators[] = {
[MAX14577_SAFEOUT] = {
.name = "SAFEOUT",
+ .of_match = of_match_ptr("SAFEOUT"),
+ .regulators_node = of_match_ptr("regulators"),
.id = MAX14577_SAFEOUT,
.ops = &max14577_safeout_ops,
.type = REGULATOR_VOLTAGE,
},
[MAX14577_CHARGER] = {
.name = "CHARGER",
+ .of_match = of_match_ptr("CHARGER"),
+ .regulators_node = of_match_ptr("regulators"),
.id = MAX14577_CHARGER,
.ops = &max14577_charger_ops,
.type = REGULATOR_CURRENT,
static const struct regulator_desc max77836_supported_regulators[] = {
[MAX14577_SAFEOUT] = {
.name = "SAFEOUT",
+ .of_match = of_match_ptr("SAFEOUT"),
+ .regulators_node = of_match_ptr("regulators"),
.id = MAX14577_SAFEOUT,
.ops = &max14577_safeout_ops,
.type = REGULATOR_VOLTAGE,
},
[MAX14577_CHARGER] = {
.name = "CHARGER",
+ .of_match = of_match_ptr("CHARGER"),
+ .regulators_node = of_match_ptr("regulators"),
.id = MAX14577_CHARGER,
.ops = &max14577_charger_ops,
.type = REGULATOR_CURRENT,
},
[MAX77836_LDO1] = {
.name = "LDO1",
+ .of_match = of_match_ptr("LDO1"),
+ .regulators_node = of_match_ptr("regulators"),
.id = MAX77836_LDO1,
.ops = &max77836_ldo_ops,
.type = REGULATOR_VOLTAGE,
},
[MAX77836_LDO2] = {
.name = "LDO2",
+ .of_match = of_match_ptr("LDO2"),
+ .regulators_node = of_match_ptr("regulators"),
.id = MAX77836_LDO2,
.ops = &max77836_ldo_ops,
.type = REGULATOR_VOLTAGE,
{ .name = "LDO2", },
};
-static int max14577_regulator_dt_parse_pdata(struct platform_device *pdev,
- enum maxim_device_type dev_type)
-{
- int ret;
- struct device_node *np;
- struct of_regulator_match *regulator_matches;
- unsigned int regulator_matches_size;
-
- np = of_get_child_by_name(pdev->dev.parent->of_node, "regulators");
- if (!np) {
- dev_err(&pdev->dev, "Failed to get child OF node for regulators\n");
- return -EINVAL;
- }
-
- switch (dev_type) {
- case MAXIM_DEVICE_TYPE_MAX77836:
- regulator_matches = max77836_regulator_matches;
- regulator_matches_size = ARRAY_SIZE(max77836_regulator_matches);
- break;
- case MAXIM_DEVICE_TYPE_MAX14577:
- default:
- regulator_matches = max14577_regulator_matches;
- regulator_matches_size = ARRAY_SIZE(max14577_regulator_matches);
- }
-
- ret = of_regulator_match(&pdev->dev, np, regulator_matches,
- regulator_matches_size);
- if (ret < 0)
- dev_err(&pdev->dev, "Error parsing regulator init data: %d\n", ret);
- else
- ret = 0;
-
- of_node_put(np);
-
- return ret;
-}
-
static inline struct regulator_init_data *match_init_data(int index,
enum maxim_device_type dev_type)
{
}
}
#else /* CONFIG_OF */
-static int max14577_regulator_dt_parse_pdata(struct platform_device *pdev,
- enum maxim_device_type dev_type)
-{
- return 0;
-}
static inline struct regulator_init_data *match_init_data(int index,
enum maxim_device_type dev_type)
{
{
struct max14577 *max14577 = dev_get_drvdata(pdev->dev.parent);
struct max14577_platform_data *pdata = dev_get_platdata(max14577->dev);
- int i, ret;
+ int i, ret = 0;
struct regulator_config config = {};
const struct regulator_desc *supported_regulators;
unsigned int supported_regulators_size;
enum maxim_device_type dev_type = max14577->dev_type;
- ret = max14577_regulator_dt_parse_pdata(pdev, dev_type);
- if (ret)
- return ret;
-
switch (dev_type) {
case MAXIM_DEVICE_TYPE_MAX77836:
supported_regulators = max77836_supported_regulators;
supported_regulators_size = ARRAY_SIZE(max14577_supported_regulators);
}
- config.dev = &pdev->dev;
+ config.dev = max14577->dev;
config.driver_data = max14577;
for (i = 0; i < supported_regulators_size; i++) {
#include <linux/bug.h>
#include <linux/err.h>
#include <linux/gpio.h>
+#include <linux/of_gpio.h>
#include <linux/slab.h>
#include <linux/platform_device.h>
#include <linux/regulator/driver.h>
#define MAX77686_DVS_MINUV 600000
#define MAX77686_DVS_UVSTEP 12500
+/*
+ * Value for configuring buck[89] and LDO{20,21,22} as GPIO control.
+ * It is the same as 'off' for other regulators.
+ */
+#define MAX77686_GPIO_CONTROL 0x0
/*
* Values used for configuring LDOs and bucks.
* Forcing low power mode: LDO1, 3-5, 9, 13, 17-26
};
struct max77686_data {
+ u64 gpio_enabled:MAX77686_REGULATORS;
+
/* Array indexed by regulator id */
unsigned int opmode[MAX77686_REGULATORS];
};
}
}
+/*
+ * When regulator is configured for GPIO control then it
+ * replaces "normal" mode. Any change from low power mode to normal
+ * should actually change to GPIO control.
+ * Map normal mode to proper value for such regulators.
+ */
+static unsigned int max77686_map_normal_mode(struct max77686_data *max77686,
+ int id)
+{
+ switch (id) {
+ case MAX77686_BUCK8:
+ case MAX77686_BUCK9:
+ case MAX77686_LDO20 ... MAX77686_LDO22:
+ if (max77686->gpio_enabled & (1 << id))
+ return MAX77686_GPIO_CONTROL;
+ }
+
+ return MAX77686_NORMAL;
+}
+
/* Some BUCKs and LDOs supports Normal[ON/OFF] mode during suspend */
static int max77686_set_suspend_disable(struct regulator_dev *rdev)
{
val = MAX77686_LDO_LOWPOWER_PWRREQ;
break;
case REGULATOR_MODE_NORMAL: /* ON in Normal Mode */
- val = MAX77686_NORMAL;
+ val = max77686_map_normal_mode(max77686, id);
break;
default:
pr_warn("%s: regulator_suspend_mode : 0x%x not supported\n",
{
unsigned int val;
struct max77686_data *max77686 = rdev_get_drvdata(rdev);
- int ret;
+ int ret, id = rdev_get_id(rdev);
switch (mode) {
case REGULATOR_MODE_STANDBY: /* switch off */
val = MAX77686_LDO_LOWPOWER_PWRREQ;
break;
case REGULATOR_MODE_NORMAL: /* ON in Normal Mode */
- val = MAX77686_NORMAL;
+ val = max77686_map_normal_mode(max77686, id);
break;
default:
pr_warn("%s: regulator_suspend_mode : 0x%x not supported\n",
if (ret)
return ret;
- max77686->opmode[rdev_get_id(rdev)] = val;
+ max77686->opmode[id] = val;
return 0;
}
shift = max77686_get_opmode_shift(id);
if (max77686->opmode[id] == MAX77686_OFF_PWRREQ)
- max77686->opmode[id] = MAX77686_NORMAL;
+ max77686->opmode[id] = max77686_map_normal_mode(max77686, id);
return regmap_update_bits(rdev->regmap, rdev->desc->enable_reg,
rdev->desc->enable_mask,
MAX77686_RAMP_RATE_MASK, ramp_value << 6);
}
+static int max77686_of_parse_cb(struct device_node *np,
+ const struct regulator_desc *desc,
+ struct regulator_config *config)
+{
+ struct max77686_data *max77686 = config->driver_data;
+
+ switch (desc->id) {
+ case MAX77686_BUCK8:
+ case MAX77686_BUCK9:
+ case MAX77686_LDO20 ... MAX77686_LDO22:
+ config->ena_gpio = of_get_named_gpio(np,
+ "maxim,ena-gpios", 0);
+ config->ena_gpio_flags = GPIOF_OUT_INIT_HIGH;
+ config->ena_gpio_initialized = true;
+ break;
+ default:
+ return 0;
+ }
+
+ if (gpio_is_valid(config->ena_gpio)) {
+ max77686->gpio_enabled |= (1 << desc->id);
+
+ return regmap_update_bits(config->regmap, desc->enable_reg,
+ desc->enable_mask,
+ MAX77686_GPIO_CONTROL);
+ }
+
+ return 0;
+}
+
static struct regulator_ops max77686_ops = {
.list_voltage = regulator_list_voltage_linear,
.map_voltage = regulator_map_voltage_linear,
.name = "LDO"#num, \
.of_match = of_match_ptr("LDO"#num), \
.regulators_node = of_match_ptr("voltage-regulators"), \
+ .of_parse_cb = max77686_of_parse_cb, \
.id = MAX77686_LDO##num, \
.ops = &max77686_ops, \
.type = REGULATOR_VOLTAGE, \
.name = "BUCK"#num, \
.of_match = of_match_ptr("BUCK"#num), \
.regulators_node = of_match_ptr("voltage-regulators"), \
+ .of_parse_cb = max77686_of_parse_cb, \
.id = MAX77686_BUCK##num, \
.ops = &max77686_ops, \
.type = REGULATOR_VOLTAGE, \
--- /dev/null
+/*
+ * max77843.c - Regulator driver for the Maxim MAX77843
+ *
+ * Copyright (C) 2015 Samsung Electronics
+ * Author: Jaewon Kim <jaewon02.kim@samsung.com>
+ * Author: Beomho Seo <beomho.seo@samsung.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/regulator/driver.h>
+#include <linux/regulator/machine.h>
+#include <linux/mfd/max77843-private.h>
+#include <linux/regulator/of_regulator.h>
+
+enum max77843_regulator_type {
+ MAX77843_SAFEOUT1 = 0,
+ MAX77843_SAFEOUT2,
+ MAX77843_CHARGER,
+
+ MAX77843_NUM,
+};
+
+static const unsigned int max77843_safeout_voltage_table[] = {
+ 4850000,
+ 4900000,
+ 4950000,
+ 3300000,
+};
+
+static int max77843_reg_is_enabled(struct regulator_dev *rdev)
+{
+ struct regmap *regmap = rdev->regmap;
+ int ret;
+ unsigned int reg;
+
+ ret = regmap_read(regmap, rdev->desc->enable_reg, ®);
+ if (ret) {
+ dev_err(&rdev->dev, "Fialed to read charger register\n");
+ return ret;
+ }
+
+ return (reg & rdev->desc->enable_mask) == rdev->desc->enable_mask;
+}
+
+static int max77843_reg_get_current_limit(struct regulator_dev *rdev)
+{
+ struct regmap *regmap = rdev->regmap;
+ unsigned int chg_min_uA = rdev->constraints->min_uA;
+ unsigned int chg_max_uA = rdev->constraints->max_uA;
+ unsigned int val;
+ int ret;
+ unsigned int reg, sel;
+
+ ret = regmap_read(regmap, MAX77843_CHG_REG_CHG_CNFG_02, ®);
+ if (ret) {
+ dev_err(&rdev->dev, "Failed to read charger register\n");
+ return ret;
+ }
+
+ sel = reg & MAX77843_CHG_FAST_CHG_CURRENT_MASK;
+
+ if (sel < 0x03)
+ sel = 0;
+ else
+ sel -= 2;
+
+ val = chg_min_uA + MAX77843_CHG_FAST_CHG_CURRENT_STEP * sel;
+ if (val > chg_max_uA)
+ return -EINVAL;
+
+ return val;
+}
+
+static int max77843_reg_set_current_limit(struct regulator_dev *rdev,
+ int min_uA, int max_uA)
+{
+ struct regmap *regmap = rdev->regmap;
+ unsigned int chg_min_uA = rdev->constraints->min_uA;
+ int sel = 0;
+
+ while (chg_min_uA + MAX77843_CHG_FAST_CHG_CURRENT_STEP * sel < min_uA)
+ sel++;
+
+ if (chg_min_uA + MAX77843_CHG_FAST_CHG_CURRENT_STEP * sel > max_uA)
+ return -EINVAL;
+
+ sel += 2;
+
+ return regmap_write(regmap, MAX77843_CHG_REG_CHG_CNFG_02, sel);
+}
+
+static struct regulator_ops max77843_charger_ops = {
+ .is_enabled = max77843_reg_is_enabled,
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .get_current_limit = max77843_reg_get_current_limit,
+ .set_current_limit = max77843_reg_set_current_limit,
+};
+
+static struct regulator_ops max77843_regulator_ops = {
+ .is_enabled = regulator_is_enabled_regmap,
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .list_voltage = regulator_list_voltage_table,
+ .get_voltage_sel = regulator_get_voltage_sel_regmap,
+ .set_voltage_sel = regulator_set_voltage_sel_regmap,
+};
+
+static const struct regulator_desc max77843_supported_regulators[] = {
+ [MAX77843_SAFEOUT1] = {
+ .name = "SAFEOUT1",
+ .id = MAX77843_SAFEOUT1,
+ .ops = &max77843_regulator_ops,
+ .of_match = of_match_ptr("SAFEOUT1"),
+ .regulators_node = of_match_ptr("regulators"),
+ .type = REGULATOR_VOLTAGE,
+ .owner = THIS_MODULE,
+ .n_voltages = ARRAY_SIZE(max77843_safeout_voltage_table),
+ .volt_table = max77843_safeout_voltage_table,
+ .enable_reg = MAX77843_SYS_REG_SAFEOUTCTRL,
+ .enable_mask = MAX77843_REG_SAFEOUTCTRL_ENSAFEOUT1,
+ .vsel_reg = MAX77843_SYS_REG_SAFEOUTCTRL,
+ .vsel_mask = MAX77843_REG_SAFEOUTCTRL_SAFEOUT1_MASK,
+ },
+ [MAX77843_SAFEOUT2] = {
+ .name = "SAFEOUT2",
+ .id = MAX77843_SAFEOUT2,
+ .ops = &max77843_regulator_ops,
+ .of_match = of_match_ptr("SAFEOUT2"),
+ .regulators_node = of_match_ptr("regulators"),
+ .type = REGULATOR_VOLTAGE,
+ .owner = THIS_MODULE,
+ .n_voltages = ARRAY_SIZE(max77843_safeout_voltage_table),
+ .volt_table = max77843_safeout_voltage_table,
+ .enable_reg = MAX77843_SYS_REG_SAFEOUTCTRL,
+ .enable_mask = MAX77843_REG_SAFEOUTCTRL_ENSAFEOUT2,
+ .vsel_reg = MAX77843_SYS_REG_SAFEOUTCTRL,
+ .vsel_mask = MAX77843_REG_SAFEOUTCTRL_SAFEOUT2_MASK,
+ },
+ [MAX77843_CHARGER] = {
+ .name = "CHARGER",
+ .id = MAX77843_CHARGER,
+ .ops = &max77843_charger_ops,
+ .of_match = of_match_ptr("CHARGER"),
+ .regulators_node = of_match_ptr("regulators"),
+ .type = REGULATOR_CURRENT,
+ .owner = THIS_MODULE,
+ .enable_reg = MAX77843_CHG_REG_CHG_CNFG_00,
+ .enable_mask = MAX77843_CHG_MASK,
+ },
+};
+
+static struct regmap *max77843_get_regmap(struct max77843 *max77843, int reg_id)
+{
+ switch (reg_id) {
+ case MAX77843_SAFEOUT1:
+ case MAX77843_SAFEOUT2:
+ return max77843->regmap;
+ case MAX77843_CHARGER:
+ return max77843->regmap_chg;
+ default:
+ return max77843->regmap;
+ }
+}
+
+static int max77843_regulator_probe(struct platform_device *pdev)
+{
+ struct max77843 *max77843 = dev_get_drvdata(pdev->dev.parent);
+ struct regulator_config config = {};
+ int i;
+
+ config.dev = max77843->dev;
+ config.driver_data = max77843;
+
+ for (i = 0; i < ARRAY_SIZE(max77843_supported_regulators); i++) {
+ struct regulator_dev *regulator;
+
+ config.regmap = max77843_get_regmap(max77843,
+ max77843_supported_regulators[i].id);
+
+ regulator = devm_regulator_register(&pdev->dev,
+ &max77843_supported_regulators[i], &config);
+ if (IS_ERR(regulator)) {
+ dev_err(&pdev->dev,
+ "Failed to regiser regulator-%d\n", i);
+ return PTR_ERR(regulator);
+ }
+ }
+
+ return 0;
+}
+
+static const struct platform_device_id max77843_regulator_id[] = {
+ { "max77843-regulator", },
+ { /* sentinel */ },
+};
+
+static struct platform_driver max77843_regulator_driver = {
+ .driver = {
+ .name = "max77843-regulator",
+ },
+ .probe = max77843_regulator_probe,
+ .id_table = max77843_regulator_id,
+};
+
+static int __init max77843_regulator_init(void)
+{
+ return platform_driver_register(&max77843_regulator_driver);
+}
+subsys_initcall(max77843_regulator_init);
+
+static void __exit max77843_regulator_exit(void)
+{
+ platform_driver_unregister(&max77843_regulator_driver);
+}
+module_exit(max77843_regulator_exit);
+
+MODULE_AUTHOR("Jaewon Kim <jaewon02.kim@samsung.com>");
+MODULE_AUTHOR("Beomho Seo <beomho.seo@samsung.com>");
+MODULE_DESCRIPTION("Maxim MAX77843 regulator driver");
+MODULE_LICENSE("GPL");
return REGULATOR_MODE_NORMAL;
}
-static struct regulator_ops max8649_dcdc_ops = {
+static const struct regulator_ops max8649_dcdc_ops = {
.set_voltage_sel = regulator_set_voltage_sel_regmap,
.get_voltage_sel = regulator_get_voltage_sel_regmap,
.list_voltage = regulator_list_voltage_linear,
.enable_is_inverted = true,
};
-static struct regmap_config max8649_regmap_config = {
+static const struct regmap_config max8649_regmap_config = {
.reg_bits = 8,
.val_bits = 8,
};
--- /dev/null
+/*
+ * Copyright (c) 2014 MediaTek Inc.
+ * Author: Flora Fu <flora.fu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+#include <linux/mfd/mt6397/core.h>
+#include <linux/mfd/mt6397/registers.h>
+#include <linux/regulator/driver.h>
+#include <linux/regulator/machine.h>
+#include <linux/regulator/mt6397-regulator.h>
+#include <linux/regulator/of_regulator.h>
+
+/*
+ * MT6397 regulators' information
+ *
+ * @desc: standard fields of regulator description.
+ * @qi: Mask for query enable signal status of regulators
+ * @vselon_reg: Register sections for hardware control mode of bucks
+ * @vselctrl_reg: Register for controlling the buck control mode.
+ * @vselctrl_mask: Mask for query buck's voltage control mode.
+ */
+struct mt6397_regulator_info {
+ struct regulator_desc desc;
+ u32 qi;
+ u32 vselon_reg;
+ u32 vselctrl_reg;
+ u32 vselctrl_mask;
+};
+
+#define MT6397_BUCK(match, vreg, min, max, step, volt_ranges, enreg, \
+ vosel, vosel_mask, voselon, vosel_ctrl) \
+[MT6397_ID_##vreg] = { \
+ .desc = { \
+ .name = #vreg, \
+ .of_match = of_match_ptr(match), \
+ .ops = &mt6397_volt_range_ops, \
+ .type = REGULATOR_VOLTAGE, \
+ .id = MT6397_ID_##vreg, \
+ .owner = THIS_MODULE, \
+ .n_voltages = (max - min)/step + 1, \
+ .linear_ranges = volt_ranges, \
+ .n_linear_ranges = ARRAY_SIZE(volt_ranges), \
+ .vsel_reg = vosel, \
+ .vsel_mask = vosel_mask, \
+ .enable_reg = enreg, \
+ .enable_mask = BIT(0), \
+ }, \
+ .qi = BIT(13), \
+ .vselon_reg = voselon, \
+ .vselctrl_reg = vosel_ctrl, \
+ .vselctrl_mask = BIT(1), \
+}
+
+#define MT6397_LDO(match, vreg, ldo_volt_table, enreg, enbit, vosel, \
+ vosel_mask) \
+[MT6397_ID_##vreg] = { \
+ .desc = { \
+ .name = #vreg, \
+ .of_match = of_match_ptr(match), \
+ .ops = &mt6397_volt_table_ops, \
+ .type = REGULATOR_VOLTAGE, \
+ .id = MT6397_ID_##vreg, \
+ .owner = THIS_MODULE, \
+ .n_voltages = ARRAY_SIZE(ldo_volt_table), \
+ .volt_table = ldo_volt_table, \
+ .vsel_reg = vosel, \
+ .vsel_mask = vosel_mask, \
+ .enable_reg = enreg, \
+ .enable_mask = BIT(enbit), \
+ }, \
+ .qi = BIT(15), \
+}
+
+#define MT6397_REG_FIXED(match, vreg, enreg, enbit, volt) \
+[MT6397_ID_##vreg] = { \
+ .desc = { \
+ .name = #vreg, \
+ .of_match = of_match_ptr(match), \
+ .ops = &mt6397_volt_fixed_ops, \
+ .type = REGULATOR_VOLTAGE, \
+ .id = MT6397_ID_##vreg, \
+ .owner = THIS_MODULE, \
+ .n_voltages = 1, \
+ .enable_reg = enreg, \
+ .enable_mask = BIT(enbit), \
+ .min_uV = volt, \
+ }, \
+ .qi = BIT(15), \
+}
+
+static const struct regulator_linear_range buck_volt_range1[] = {
+ REGULATOR_LINEAR_RANGE(700000, 0, 0x7f, 6250),
+};
+
+static const struct regulator_linear_range buck_volt_range2[] = {
+ REGULATOR_LINEAR_RANGE(800000, 0, 0x7f, 6250),
+};
+
+static const struct regulator_linear_range buck_volt_range3[] = {
+ REGULATOR_LINEAR_RANGE(1500000, 0, 0x1f, 20000),
+};
+
+static const u32 ldo_volt_table1[] = {
+ 1500000, 1800000, 2500000, 2800000,
+};
+
+static const u32 ldo_volt_table2[] = {
+ 1800000, 3300000,
+};
+
+static const u32 ldo_volt_table3[] = {
+ 3000000, 3300000,
+};
+
+static const u32 ldo_volt_table4[] = {
+ 1220000, 1300000, 1500000, 1800000, 2500000, 2800000, 3000000, 3300000,
+};
+
+static const u32 ldo_volt_table5[] = {
+ 1200000, 1300000, 1500000, 1800000, 2500000, 2800000, 3000000, 3300000,
+};
+
+static const u32 ldo_volt_table5_v2[] = {
+ 1200000, 1000000, 1500000, 1800000, 2500000, 2800000, 3000000, 3300000,
+};
+
+static const u32 ldo_volt_table6[] = {
+ 1200000, 1300000, 1500000, 1800000, 2500000, 2800000, 3000000, 2000000,
+};
+
+static const u32 ldo_volt_table7[] = {
+ 1300000, 1500000, 1800000, 2000000, 2500000, 2800000, 3000000, 3300000,
+};
+
+static int mt6397_get_status(struct regulator_dev *rdev)
+{
+ int ret;
+ u32 regval;
+ struct mt6397_regulator_info *info = rdev_get_drvdata(rdev);
+
+ ret = regmap_read(rdev->regmap, info->desc.enable_reg, ®val);
+ if (ret != 0) {
+ dev_err(&rdev->dev, "Failed to get enable reg: %d\n", ret);
+ return ret;
+ }
+
+ return (regval & info->qi) ? REGULATOR_STATUS_ON : REGULATOR_STATUS_OFF;
+}
+
+static struct regulator_ops mt6397_volt_range_ops = {
+ .list_voltage = regulator_list_voltage_linear_range,
+ .map_voltage = regulator_map_voltage_linear_range,
+ .set_voltage_sel = regulator_set_voltage_sel_regmap,
+ .get_voltage_sel = regulator_get_voltage_sel_regmap,
+ .set_voltage_time_sel = regulator_set_voltage_time_sel,
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .is_enabled = regulator_is_enabled_regmap,
+ .get_status = mt6397_get_status,
+};
+
+static struct regulator_ops mt6397_volt_table_ops = {
+ .list_voltage = regulator_list_voltage_table,
+ .map_voltage = regulator_map_voltage_iterate,
+ .set_voltage_sel = regulator_set_voltage_sel_regmap,
+ .get_voltage_sel = regulator_get_voltage_sel_regmap,
+ .set_voltage_time_sel = regulator_set_voltage_time_sel,
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .is_enabled = regulator_is_enabled_regmap,
+ .get_status = mt6397_get_status,
+};
+
+static struct regulator_ops mt6397_volt_fixed_ops = {
+ .list_voltage = regulator_list_voltage_linear,
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .is_enabled = regulator_is_enabled_regmap,
+ .get_status = mt6397_get_status,
+};
+
+/* The array is indexed by id(MT6397_ID_XXX) */
+static struct mt6397_regulator_info mt6397_regulators[] = {
+ MT6397_BUCK("buck_vpca15", VPCA15, 700000, 1493750, 6250,
+ buck_volt_range1, MT6397_VCA15_CON7, MT6397_VCA15_CON9, 0x7f,
+ MT6397_VCA15_CON10, MT6397_VCA15_CON5),
+ MT6397_BUCK("buck_vpca7", VPCA7, 700000, 1493750, 6250,
+ buck_volt_range1, MT6397_VPCA7_CON7, MT6397_VPCA7_CON9, 0x7f,
+ MT6397_VPCA7_CON10, MT6397_VPCA7_CON5),
+ MT6397_BUCK("buck_vsramca15", VSRAMCA15, 700000, 1493750, 6250,
+ buck_volt_range1, MT6397_VSRMCA15_CON7, MT6397_VSRMCA15_CON9,
+ 0x7f, MT6397_VSRMCA15_CON10, MT6397_VSRMCA15_CON5),
+ MT6397_BUCK("buck_vsramca7", VSRAMCA7, 700000, 1493750, 6250,
+ buck_volt_range1, MT6397_VSRMCA7_CON7, MT6397_VSRMCA7_CON9,
+ 0x7f, MT6397_VSRMCA7_CON10, MT6397_VSRMCA7_CON5),
+ MT6397_BUCK("buck_vcore", VCORE, 700000, 1493750, 6250,
+ buck_volt_range1, MT6397_VCORE_CON7, MT6397_VCORE_CON9, 0x7f,
+ MT6397_VCORE_CON10, MT6397_VCORE_CON5),
+ MT6397_BUCK("buck_vgpu", VGPU, 700000, 1493750, 6250, buck_volt_range1,
+ MT6397_VGPU_CON7, MT6397_VGPU_CON9, 0x7f,
+ MT6397_VGPU_CON10, MT6397_VGPU_CON5),
+ MT6397_BUCK("buck_vdrm", VDRM, 800000, 1593750, 6250, buck_volt_range2,
+ MT6397_VDRM_CON7, MT6397_VDRM_CON9, 0x7f,
+ MT6397_VDRM_CON10, MT6397_VDRM_CON5),
+ MT6397_BUCK("buck_vio18", VIO18, 1500000, 2120000, 20000,
+ buck_volt_range3, MT6397_VIO18_CON7, MT6397_VIO18_CON9, 0x1f,
+ MT6397_VIO18_CON10, MT6397_VIO18_CON5),
+ MT6397_REG_FIXED("ldo_vtcxo", VTCXO, MT6397_ANALDO_CON0, 10, 2800000),
+ MT6397_REG_FIXED("ldo_va28", VA28, MT6397_ANALDO_CON1, 14, 2800000),
+ MT6397_LDO("ldo_vcama", VCAMA, ldo_volt_table1,
+ MT6397_ANALDO_CON2, 15, MT6397_ANALDO_CON6, 0xC0),
+ MT6397_REG_FIXED("ldo_vio28", VIO28, MT6397_DIGLDO_CON0, 14, 2800000),
+ MT6397_REG_FIXED("ldo_vusb", VUSB, MT6397_DIGLDO_CON1, 14, 3300000),
+ MT6397_LDO("ldo_vmc", VMC, ldo_volt_table2,
+ MT6397_DIGLDO_CON2, 12, MT6397_DIGLDO_CON29, 0x10),
+ MT6397_LDO("ldo_vmch", VMCH, ldo_volt_table3,
+ MT6397_DIGLDO_CON3, 14, MT6397_DIGLDO_CON17, 0x80),
+ MT6397_LDO("ldo_vemc3v3", VEMC3V3, ldo_volt_table3,
+ MT6397_DIGLDO_CON4, 14, MT6397_DIGLDO_CON18, 0x10),
+ MT6397_LDO("ldo_vgp1", VGP1, ldo_volt_table4,
+ MT6397_DIGLDO_CON5, 15, MT6397_DIGLDO_CON19, 0xE0),
+ MT6397_LDO("ldo_vgp2", VGP2, ldo_volt_table5,
+ MT6397_DIGLDO_CON6, 15, MT6397_DIGLDO_CON20, 0xE0),
+ MT6397_LDO("ldo_vgp3", VGP3, ldo_volt_table5,
+ MT6397_DIGLDO_CON7, 15, MT6397_DIGLDO_CON21, 0xE0),
+ MT6397_LDO("ldo_vgp4", VGP4, ldo_volt_table5,
+ MT6397_DIGLDO_CON8, 15, MT6397_DIGLDO_CON22, 0xE0),
+ MT6397_LDO("ldo_vgp5", VGP5, ldo_volt_table6,
+ MT6397_DIGLDO_CON9, 15, MT6397_DIGLDO_CON23, 0xE0),
+ MT6397_LDO("ldo_vgp6", VGP6, ldo_volt_table5,
+ MT6397_DIGLDO_CON10, 15, MT6397_DIGLDO_CON33, 0xE0),
+ MT6397_LDO("ldo_vibr", VIBR, ldo_volt_table7,
+ MT6397_DIGLDO_CON24, 15, MT6397_DIGLDO_CON25, 0xE00),
+};
+
+static int mt6397_set_buck_vosel_reg(struct platform_device *pdev)
+{
+ struct mt6397_chip *mt6397 = dev_get_drvdata(pdev->dev.parent);
+ int i;
+ u32 regval;
+
+ for (i = 0; i < MT6397_MAX_REGULATOR; i++) {
+ if (mt6397_regulators[i].vselctrl_reg) {
+ if (regmap_read(mt6397->regmap,
+ mt6397_regulators[i].vselctrl_reg,
+ ®val) < 0) {
+ dev_err(&pdev->dev,
+ "Failed to read buck ctrl\n");
+ return -EIO;
+ }
+
+ if (regval & mt6397_regulators[i].vselctrl_mask) {
+ mt6397_regulators[i].desc.vsel_reg =
+ mt6397_regulators[i].vselon_reg;
+ }
+ }
+ }
+
+ return 0;
+}
+
+static int mt6397_regulator_probe(struct platform_device *pdev)
+{
+ struct mt6397_chip *mt6397 = dev_get_drvdata(pdev->dev.parent);
+ struct regulator_config config = {};
+ struct regulator_dev *rdev;
+ int i;
+ u32 reg_value, version;
+
+ /* Query buck controller to select activated voltage register part */
+ if (mt6397_set_buck_vosel_reg(pdev))
+ return -EIO;
+
+ /* Read PMIC chip revision to update constraints and voltage table */
+ if (regmap_read(mt6397->regmap, MT6397_CID, ®_value) < 0) {
+ dev_err(&pdev->dev, "Failed to read Chip ID\n");
+ return -EIO;
+ }
+ dev_info(&pdev->dev, "Chip ID = 0x%x\n", reg_value);
+
+ version = (reg_value & 0xFF);
+ switch (version) {
+ case MT6397_REGULATOR_ID91:
+ mt6397_regulators[MT6397_ID_VGP2].desc.volt_table =
+ ldo_volt_table5_v2;
+ break;
+ default:
+ break;
+ }
+
+ for (i = 0; i < MT6397_MAX_REGULATOR; i++) {
+ config.dev = &pdev->dev;
+ config.driver_data = &mt6397_regulators[i];
+ config.regmap = mt6397->regmap;
+ rdev = devm_regulator_register(&pdev->dev,
+ &mt6397_regulators[i].desc, &config);
+ if (IS_ERR(rdev)) {
+ dev_err(&pdev->dev, "failed to register %s\n",
+ mt6397_regulators[i].desc.name);
+ return PTR_ERR(rdev);
+ }
+ }
+
+ return 0;
+}
+
+static struct platform_driver mt6397_regulator_driver = {
+ .driver = {
+ .name = "mt6397-regulator",
+ },
+ .probe = mt6397_regulator_probe,
+};
+
+module_platform_driver(mt6397_regulator_driver);
+
+MODULE_AUTHOR("Flora Fu <flora.fu@mediatek.com>");
+MODULE_DESCRIPTION("Regulator Driver for MediaTek MT6397 PMIC");
+MODULE_LICENSE("GPL");
+MODULE_ALIAS("platform:mt6397-regulator");
struct regulator_init_data *regulator_of_get_init_data(struct device *dev,
const struct regulator_desc *desc,
+ struct regulator_config *config,
struct device_node **node)
{
struct device_node *search, *child;
break;
}
+ if (desc->of_parse_cb) {
+ if (desc->of_parse_cb(child, desc, config)) {
+ dev_err(dev,
+ "driver callback failed to parse DT for regulator %s\n",
+ child->name);
+ init_data = NULL;
+ break;
+ }
+ }
+
of_node_get(child);
*node = child;
break;
#define PFUZE100_VGEN5VOL 0x70
#define PFUZE100_VGEN6VOL 0x71
-enum chips { PFUZE100, PFUZE200 };
+enum chips { PFUZE100, PFUZE200, PFUZE3000 = 3 };
struct pfuze_regulator {
struct regulator_desc desc;
1000000, 1100000, 1200000, 1300000, 1500000, 1800000, 3000000,
};
+static const int pfuze3000_sw2lo[] = {
+ 1500000, 1550000, 1600000, 1650000, 1700000, 1750000, 1800000, 1850000,
+};
+
+static const int pfuze3000_sw2hi[] = {
+ 2500000, 2800000, 2850000, 3000000, 3100000, 3150000, 3200000, 3300000,
+};
+
static const struct i2c_device_id pfuze_device_id[] = {
{.name = "pfuze100", .driver_data = PFUZE100},
{.name = "pfuze200", .driver_data = PFUZE200},
+ {.name = "pfuze3000", .driver_data = PFUZE3000},
{ }
};
MODULE_DEVICE_TABLE(i2c, pfuze_device_id);
static const struct of_device_id pfuze_dt_ids[] = {
{ .compatible = "fsl,pfuze100", .data = (void *)PFUZE100},
{ .compatible = "fsl,pfuze200", .data = (void *)PFUZE200},
+ { .compatible = "fsl,pfuze3000", .data = (void *)PFUZE3000},
{ }
};
MODULE_DEVICE_TABLE(of, pfuze_dt_ids);
.stby_mask = 0x20, \
}
+#define PFUZE3000_VCC_REG(_chip, _name, base, min, max, step) { \
+ .desc = { \
+ .name = #_name, \
+ .n_voltages = ((max) - (min)) / (step) + 1, \
+ .ops = &pfuze100_ldo_regulator_ops, \
+ .type = REGULATOR_VOLTAGE, \
+ .id = _chip ## _ ## _name, \
+ .owner = THIS_MODULE, \
+ .min_uV = (min), \
+ .uV_step = (step), \
+ .vsel_reg = (base), \
+ .vsel_mask = 0x3, \
+ .enable_reg = (base), \
+ .enable_mask = 0x10, \
+ }, \
+ .stby_reg = (base), \
+ .stby_mask = 0x20, \
+}
+
+
+#define PFUZE3000_SW2_REG(_chip, _name, base, min, max, step) { \
+ .desc = { \
+ .name = #_name,\
+ .n_voltages = ((max) - (min)) / (step) + 1, \
+ .ops = &pfuze100_sw_regulator_ops, \
+ .type = REGULATOR_VOLTAGE, \
+ .id = _chip ## _ ## _name, \
+ .owner = THIS_MODULE, \
+ .min_uV = (min), \
+ .uV_step = (step), \
+ .vsel_reg = (base) + PFUZE100_VOL_OFFSET, \
+ .vsel_mask = 0x7, \
+ }, \
+ .stby_reg = (base) + PFUZE100_STANDBY_OFFSET, \
+ .stby_mask = 0x7, \
+}
+
+#define PFUZE3000_SW3_REG(_chip, _name, base, min, max, step) { \
+ .desc = { \
+ .name = #_name,\
+ .n_voltages = ((max) - (min)) / (step) + 1, \
+ .ops = &pfuze100_sw_regulator_ops, \
+ .type = REGULATOR_VOLTAGE, \
+ .id = _chip ## _ ## _name, \
+ .owner = THIS_MODULE, \
+ .min_uV = (min), \
+ .uV_step = (step), \
+ .vsel_reg = (base) + PFUZE100_VOL_OFFSET, \
+ .vsel_mask = 0xf, \
+ }, \
+ .stby_reg = (base) + PFUZE100_STANDBY_OFFSET, \
+ .stby_mask = 0xf, \
+}
+
/* PFUZE100 */
static struct pfuze_regulator pfuze100_regulators[] = {
PFUZE100_SW_REG(PFUZE100, SW1AB, PFUZE100_SW1ABVOL, 300000, 1875000, 25000),
PFUZE100_VGEN_REG(PFUZE200, VGEN6, PFUZE100_VGEN6VOL, 1800000, 3300000, 100000),
};
+static struct pfuze_regulator pfuze3000_regulators[] = {
+ PFUZE100_SW_REG(PFUZE3000, SW1A, PFUZE100_SW1ABVOL, 700000, 1475000, 25000),
+ PFUZE100_SW_REG(PFUZE3000, SW1B, PFUZE100_SW1CVOL, 700000, 1475000, 25000),
+ PFUZE100_SWB_REG(PFUZE3000, SW2, PFUZE100_SW2VOL, 0x7, pfuze3000_sw2lo),
+ PFUZE3000_SW3_REG(PFUZE3000, SW3, PFUZE100_SW3AVOL, 900000, 1650000, 50000),
+ PFUZE100_SWB_REG(PFUZE3000, SWBST, PFUZE100_SWBSTCON1, 0x3, pfuze100_swbst),
+ PFUZE100_SWB_REG(PFUZE3000, VSNVS, PFUZE100_VSNVSVOL, 0x7, pfuze100_vsnvs),
+ PFUZE100_FIXED_REG(PFUZE3000, VREFDDR, PFUZE100_VREFDDRCON, 750000),
+ PFUZE100_VGEN_REG(PFUZE3000, VLDO1, PFUZE100_VGEN1VOL, 1800000, 3300000, 100000),
+ PFUZE100_VGEN_REG(PFUZE3000, VLDO2, PFUZE100_VGEN2VOL, 800000, 1550000, 50000),
+ PFUZE3000_VCC_REG(PFUZE3000, VCCSD, PFUZE100_VGEN3VOL, 2850000, 3300000, 150000),
+ PFUZE3000_VCC_REG(PFUZE3000, V33, PFUZE100_VGEN4VOL, 2850000, 3300000, 150000),
+ PFUZE100_VGEN_REG(PFUZE3000, VLDO3, PFUZE100_VGEN5VOL, 1800000, 3300000, 100000),
+ PFUZE100_VGEN_REG(PFUZE3000, VLDO4, PFUZE100_VGEN6VOL, 1800000, 3300000, 100000),
+};
+
static struct pfuze_regulator *pfuze_regulators;
#ifdef CONFIG_OF
{ .name = "vgen6", },
};
+/* PFUZE3000 */
+static struct of_regulator_match pfuze3000_matches[] = {
+
+ { .name = "sw1a", },
+ { .name = "sw1b", },
+ { .name = "sw2", },
+ { .name = "sw3", },
+ { .name = "swbst", },
+ { .name = "vsnvs", },
+ { .name = "vrefddr", },
+ { .name = "vldo1", },
+ { .name = "vldo2", },
+ { .name = "vccsd", },
+ { .name = "v33", },
+ { .name = "vldo3", },
+ { .name = "vldo4", },
+};
+
static struct of_regulator_match *pfuze_matches;
static int pfuze_parse_regulators_dt(struct pfuze_chip *chip)
}
switch (chip->chip_id) {
+ case PFUZE3000:
+ pfuze_matches = pfuze3000_matches;
+ ret = of_regulator_match(dev, parent, pfuze3000_matches,
+ ARRAY_SIZE(pfuze3000_matches));
+ break;
case PFUZE200:
pfuze_matches = pfuze200_matches;
ret = of_regulator_match(dev, parent, pfuze200_matches,
* as ID=8 in PFUZE100
*/
dev_info(pfuze_chip->dev, "Assuming misprogrammed ID=0x8");
- } else if ((value & 0x0f) != pfuze_chip->chip_id) {
+ } else if ((value & 0x0f) != pfuze_chip->chip_id &&
+ (value & 0xf0) >> 4 != pfuze_chip->chip_id) {
/* device id NOT match with your setting */
dev_warn(pfuze_chip->dev, "Illegal ID: %x\n", value);
return -ENODEV;
int i, ret;
const struct of_device_id *match;
u32 regulator_num;
- u32 sw_check_start, sw_check_end;
+ u32 sw_check_start, sw_check_end, sw_hi = 0x40;
pfuze_chip = devm_kzalloc(&client->dev, sizeof(*pfuze_chip),
GFP_KERNEL);
/* use the right regulators after identify the right device */
switch (pfuze_chip->chip_id) {
+ case PFUZE3000:
+ pfuze_regulators = pfuze3000_regulators;
+ regulator_num = ARRAY_SIZE(pfuze3000_regulators);
+ sw_check_start = PFUZE3000_SW2;
+ sw_check_end = PFUZE3000_SW2;
+ sw_hi = 1 << 3;
+ break;
case PFUZE200:
pfuze_regulators = pfuze200_regulators;
regulator_num = ARRAY_SIZE(pfuze200_regulators);
sw_check_start = PFUZE200_SW2;
sw_check_end = PFUZE200_SW3B;
break;
-
case PFUZE100:
default:
pfuze_regulators = pfuze100_regulators;
break;
}
dev_info(&client->dev, "pfuze%s found.\n",
- (pfuze_chip->chip_id == PFUZE100) ? "100" : "200");
+ (pfuze_chip->chip_id == PFUZE100) ? "100" :
+ ((pfuze_chip->chip_id == PFUZE200) ? "200" : "3000"));
memcpy(pfuze_chip->regulator_descs, pfuze_regulators,
sizeof(pfuze_chip->regulator_descs));
/* SW2~SW4 high bit check and modify the voltage value table */
if (i >= sw_check_start && i <= sw_check_end) {
regmap_read(pfuze_chip->regmap, desc->vsel_reg, &val);
- if (val & 0x40) {
- desc->min_uV = 800000;
- desc->uV_step = 50000;
- desc->n_voltages = 51;
+ if (val & sw_hi) {
+ if (pfuze_chip->chip_id == PFUZE3000) {
+ desc->volt_table = pfuze3000_sw2hi;
+ desc->n_voltages = ARRAY_SIZE(pfuze3000_sw2hi);
+ } else {
+ desc->min_uV = 800000;
+ desc->uV_step = 50000;
+ desc->n_voltages = 51;
+ }
}
}
return uV;
mutex_lock(&vreg->lock);
- vreg->uV = uV;
if (vreg->is_enabled)
- ret = rpm_reg_write(vreg, req, vreg->uV / 1000);
+ ret = rpm_reg_write(vreg, req, uV / 1000);
+
+ if (!ret)
+ vreg->uV = uV;
mutex_unlock(&vreg->lock);
return ret;
return uV;
mutex_lock(&vreg->lock);
- vreg->uV = uV;
if (vreg->is_enabled)
- ret = rpm_reg_write(vreg, req, vreg->uV);
+ ret = rpm_reg_write(vreg, req, uV);
+
+ if (!ret)
+ vreg->uV = uV;
mutex_unlock(&vreg->lock);
return ret;
vreg->desc.owner = THIS_MODULE;
vreg->desc.type = REGULATOR_VOLTAGE;
vreg->desc.name = pdev->dev.of_node->name;
+ vreg->desc.supply_name = "vin";
vreg->rpm = dev_get_drvdata(pdev->dev.parent);
if (!vreg->rpm) {
break;
}
- if (force_mode < 0) {
+ if (force_mode == -1) {
dev_err(&pdev->dev, "invalid force mode\n");
return -EINVAL;
}
RK808_RAMP_RATE_MASK, ramp_value);
}
-int rk808_set_suspend_voltage(struct regulator_dev *rdev, int uv)
+static int rk808_set_suspend_voltage(struct regulator_dev *rdev, int uv)
{
unsigned int reg;
int sel = regulator_map_voltage_linear_range(rdev, uv, uv);
sel);
}
-int rk808_set_suspend_enable(struct regulator_dev *rdev)
+static int rk808_set_suspend_enable(struct regulator_dev *rdev)
{
unsigned int reg;
0);
}
-int rk808_set_suspend_disable(struct regulator_dev *rdev)
+static int rk808_set_suspend_disable(struct regulator_dev *rdev)
{
unsigned int reg;
static const struct regulator_desc rt5033_supported_regulators[] = {
[RT5033_BUCK] = {
.name = "BUCK",
+ .of_match = of_match_ptr("BUCK"),
+ .regulators_node = of_match_ptr("regulators"),
.id = RT5033_BUCK,
.ops = &rt5033_buck_ops,
.type = REGULATOR_VOLTAGE,
},
[RT5033_LDO] = {
.name = "LDO",
+ .of_match = of_match_ptr("LDO"),
+ .regulators_node = of_match_ptr("regulators"),
.id = RT5033_LDO,
.ops = &rt5033_buck_ops,
.type = REGULATOR_VOLTAGE,
},
[RT5033_SAFE_LDO] = {
.name = "SAFE_LDO",
+ .of_match = of_match_ptr("SAFE_LDO"),
+ .regulators_node = of_match_ptr("regulators"),
.id = RT5033_SAFE_LDO,
.ops = &rt5033_safe_ldo_ops,
.type = REGULATOR_VOLTAGE,
int ret, i;
struct regulator_config config = {};
- config.dev = &pdev->dev;
+ config.dev = rt5033->dev;
config.driver_data = rt5033;
for (i = 0; i < ARRAY_SIZE(rt5033_supported_regulators); i++) {
}
/* Operations permitted on VDCDCx */
-static struct regulator_ops tps65023_dcdc_ops = {
+static const struct regulator_ops tps65023_dcdc_ops = {
.is_enabled = regulator_is_enabled_regmap,
.enable = regulator_enable_regmap,
.disable = regulator_disable_regmap,
};
/* Operations permitted on LDOx */
-static struct regulator_ops tps65023_ldo_ops = {
+static const struct regulator_ops tps65023_ldo_ops = {
.is_enabled = regulator_is_enabled_regmap,
.enable = regulator_enable_regmap,
.disable = regulator_disable_regmap,
.map_voltage = regulator_map_voltage_ascend,
};
-static struct regmap_config tps65023_regmap_config = {
+static const struct regmap_config tps65023_regmap_config = {
.reg_bits = 8,
.val_bits = 8,
};
{
int err = -ENODEV;
struct rtc_time tm;
- struct timespec tv = {
+ struct timespec64 tv64 = {
.tv_nsec = NSEC_PER_SEC >> 1,
};
struct rtc_device *rtc = rtc_class_open(CONFIG_RTC_HCTOSYS_DEVICE);
}
- err = rtc_valid_tm(&tm);
- if (err) {
- dev_err(rtc->dev.parent,
- "hctosys: invalid date/time\n");
- goto err_invalid;
- }
-
- rtc_tm_to_time(&tm, &tv.tv_sec);
+ tv64.tv_sec = rtc_tm_to_time64(&tm);
- err = do_settimeofday(&tv);
+ err = do_settimeofday64(&tv64);
dev_info(rtc->dev.parent,
"setting system clock to "
- "%d-%02d-%02d %02d:%02d:%02d UTC (%u)\n",
+ "%d-%02d-%02d %02d:%02d:%02d UTC (%lld)\n",
tm.tm_year + 1900, tm.tm_mon + 1, tm.tm_mday,
tm.tm_hour, tm.tm_min, tm.tm_sec,
- (unsigned int) tv.tv_sec);
+ (long long) tv64.tv_sec);
-err_invalid:
err_read:
rtc_class_close(rtc);
else if (rtc->ops->set_time)
err = rtc->ops->set_time(rtc->dev.parent, tm);
else if (rtc->ops->set_mmss) {
- unsigned long secs;
- err = rtc_tm_to_time(tm, &secs);
- if (err == 0)
- err = rtc->ops->set_mmss(rtc->dev.parent, secs);
+ time64_t secs64 = rtc_tm_to_time64(tm);
+ err = rtc->ops->set_mmss(rtc->dev.parent, secs64);
} else
err = -EINVAL;
err = rtc->ops->read_time(rtc->dev.parent, &old);
if (err == 0) {
- rtc_time_to_tm(secs, &new);
+ rtc_time64_to_tm(secs, &new);
/*
* avoid writing when we're going to change the day of
int err;
struct rtc_time before, now;
int first_time = 1;
- unsigned long t_now, t_alm;
+ time64_t t_now, t_alm;
enum { none, day, month, year } missing = none;
unsigned days;
}
/* with luck, no rollover is needed */
- rtc_tm_to_time(&now, &t_now);
- rtc_tm_to_time(&alarm->time, &t_alm);
+ t_now = rtc_tm_to_time64(&now);
+ t_alm = rtc_tm_to_time64(&alarm->time);
if (t_now < t_alm)
goto done;
case day:
dev_dbg(&rtc->dev, "alarm rollover: %s\n", "day");
t_alm += 24 * 60 * 60;
- rtc_time_to_tm(t_alm, &alarm->time);
+ rtc_time64_to_tm(t_alm, &alarm->time);
break;
/* Month rollover ... if it's the 31th, an alarm on the 3rd will
static int __rtc_set_alarm(struct rtc_device *rtc, struct rtc_wkalrm *alarm)
{
struct rtc_time tm;
- long now, scheduled;
+ time64_t now, scheduled;
int err;
err = rtc_valid_tm(&alarm->time);
if (err)
return err;
- rtc_tm_to_time(&alarm->time, &scheduled);
+ scheduled = rtc_tm_to_time64(&alarm->time);
/* Make sure we're not setting alarms in the past */
err = __rtc_read_time(rtc, &tm);
if (err)
return err;
- rtc_tm_to_time(&tm, &now);
+ now = rtc_tm_to_time64(&tm);
if (scheduled <= now)
return -ETIME;
/*
* Not supported here.
*/
{
- unsigned long now, then;
+ time64_t now, then;
err = rtc_read_time(rtc, &tm);
if (err < 0)
return err;
- rtc_tm_to_time(&tm, &now);
+ now = rtc_tm_to_time64(&tm);
alarm.time.tm_mday = tm.tm_mday;
alarm.time.tm_mon = tm.tm_mon;
err = rtc_valid_tm(&alarm.time);
if (err < 0)
return err;
- rtc_tm_to_time(&alarm.time, &then);
+ then = rtc_tm_to_time64(&alarm.time);
/* alarm may need to wrap into tomorrow */
if (then < now) {
- rtc_time_to_tm(now + 24 * 60 * 60, &tm);
+ rtc_time64_to_tm(now + 24 * 60 * 60, &tm);
alarm.time.tm_mday = tm.tm_mday;
alarm.time.tm_mon = tm.tm_mon;
alarm.time.tm_year = tm.tm_year;
*
* If temporary failure is indicated the caller should try again 'soon'
*/
-int rtc_set_ntp_time(struct timespec now)
+int rtc_set_ntp_time(struct timespec64 now)
{
struct rtc_device *rtc;
struct rtc_time tm;
int err = -ENODEV;
if (now.tv_nsec < (NSEC_PER_SEC >> 1))
- rtc_time_to_tm(now.tv_sec, &tm);
+ rtc_time64_to_tm(now.tv_sec, &tm);
else
- rtc_time_to_tm(now.tv_sec + 1, &tm);
+ rtc_time64_to_tm(now.tv_sec + 1, &tm);
rtc = rtc_class_open(CONFIG_RTC_HCTOSYS_DEVICE);
if (rtc) {
struct scsi_device_handler *scsi_dh = scsi_dh_data->scsi_dh;
struct scsi_device *sdev = scsi_dh_data->sdev;
+ scsi_dh->detach(sdev);
+
spin_lock_irq(sdev->request_queue->queue_lock);
sdev->scsi_dh_data = NULL;
spin_unlock_irq(sdev->request_queue->queue_lock);
- scsi_dh->detach(sdev);
sdev_printk(KERN_NOTICE, sdev, "%s: Detached\n", scsi_dh->name);
module_put(scsi_dh->module);
}
*/
sd_set_flush_flag(sdkp);
- max_xfer = min_not_zero(queue_max_hw_sectors(sdkp->disk->queue),
- sdkp->max_xfer_blocks);
+ max_xfer = sdkp->max_xfer_blocks;
max_xfer <<= ilog2(sdp->sector_size) - 9;
+
+ max_xfer = min_not_zero(queue_max_hw_sectors(sdkp->disk->queue),
+ max_xfer);
blk_queue_max_hw_sectors(sdkp->disk->queue, max_xfer);
set_capacity(disk, sdkp->capacity);
sd_config_write_same(sdkp);
help
SPI master controller for DaVinci/DA8x/OMAP-L/AM1x SPI modules.
+config SPI_DLN2
+ tristate "Diolan DLN-2 USB SPI adapter"
+ depends on MFD_DLN2
+ help
+ If you say yes to this option, support will be included for Diolan
+ DLN2, a USB to SPI interface.
+
+ This driver can also be built as a module. If so, the module
+ will be called spi-dln2.
+
config SPI_EFM32
tristate "EFM32 SPI controller"
depends on OF && ARM && (ARCH_EFM32 || COMPILE_TEST)
depends on FSL_SOC
config SPI_FSL_SPI
- bool "Freescale SPI controller and Aeroflex Gaisler GRLIB SPI controller"
+ tristate "Freescale SPI controller and Aeroflex Gaisler GRLIB SPI controller"
depends on OF
select SPI_FSL_LIB
select SPI_FSL_CPM if FSL_SOC
config SPI_FSL_DSPI
tristate "Freescale DSPI controller"
- select SPI_BITBANG
select REGMAP_MMIO
depends on SOC_VF610 || COMPILE_TEST
help
mode. VF610 platform uses the controller.
config SPI_FSL_ESPI
- bool "Freescale eSPI controller"
+ tristate "Freescale eSPI controller"
depends on FSL_SOC
select SPI_FSL_LIB
help
config SPI_S3C64XX
tristate "Samsung S3C64XX series type SPI"
depends on (PLAT_SAMSUNG || ARCH_EXYNOS)
- select S3C64XX_PL080 if ARCH_S3C64XX
help
SPI driver for Samsung S3C64XX and newer SoCs.
help
SPI driver for CSR SiRFprimaII SoCs
+config SPI_ST_SSC4
+ tristate "STMicroelectronics SPI SSC-based driver"
+ depends on ARCH_STI
+ help
+ STMicroelectronics SoCs support for SPI. If you say yes to
+ this option, support will be included for the SSC driven SPI.
+
config SPI_SUN4I
tristate "Allwinner A10 SoCs SPI controller"
depends on ARCH_SUNXI || COMPILE_TEST
16 bit words in SPI mode 0, automatically asserting CS on transfer
start and deasserting on end.
-
config SPI_NUC900
tristate "Nuvoton NUC900 series SPI"
depends on ARCH_W90X900
obj-$(CONFIG_SPI_CLPS711X) += spi-clps711x.o
obj-$(CONFIG_SPI_COLDFIRE_QSPI) += spi-coldfire-qspi.o
obj-$(CONFIG_SPI_DAVINCI) += spi-davinci.o
+obj-$(CONFIG_SPI_DLN2) += spi-dln2.o
obj-$(CONFIG_SPI_DESIGNWARE) += spi-dw.o
obj-$(CONFIG_SPI_DW_MMIO) += spi-dw-mmio.o
obj-$(CONFIG_SPI_DW_PCI) += spi-dw-midpci.o
obj-$(CONFIG_SPI_SH_MSIOF) += spi-sh-msiof.o
obj-$(CONFIG_SPI_SH_SCI) += spi-sh-sci.o
obj-$(CONFIG_SPI_SIRF) += spi-sirf.o
+obj-$(CONFIG_SPI_ST_SSC4) += spi-st-ssc4.o
obj-$(CONFIG_SPI_SUN4I) += spi-sun4i.o
obj-$(CONFIG_SPI_SUN6I) += spi-sun6i.o
obj-$(CONFIG_SPI_TEGRA114) += spi-tegra114.o
struct atmel_spi_device *asd;
int timeout;
int ret;
+ unsigned long dma_timeout;
as = spi_master_get_devdata(master);
/* interrupts are disabled, so free the lock for schedule */
atmel_spi_unlock(as);
- ret = wait_for_completion_timeout(&as->xfer_completion,
- SPI_DMA_TIMEOUT);
+ dma_timeout = wait_for_completion_timeout(&as->xfer_completion,
+ SPI_DMA_TIMEOUT);
atmel_spi_lock(as);
- if (WARN_ON(ret == 0)) {
- dev_err(&spi->dev,
- "spi trasfer timeout, err %d\n", ret);
+ if (WARN_ON(dma_timeout == 0)) {
+ dev_err(&spi->dev, "spi transfer timeout\n");
as->done_status = -EIO;
- } else {
- ret = 0;
}
if (as->done_status)
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/init.h>
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
*/
#include <linux/clk.h>
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the
- * Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
*/
#include <linux/kernel.h>
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/spinlock.h>
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
#include <linux/kernel.h>
#include <linux/init.h>
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA
- *
*/
#include <linux/kernel.h>
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/interrupt.h>
--- /dev/null
+/*
+ * Driver for the Diolan DLN-2 USB-SPI adapter
+ *
+ * Copyright (c) 2014 Intel Corporation
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation, version 2.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/mfd/dln2.h>
+#include <linux/spi/spi.h>
+#include <linux/pm_runtime.h>
+#include <asm/unaligned.h>
+
+#define DLN2_SPI_MODULE_ID 0x02
+#define DLN2_SPI_CMD(cmd) DLN2_CMD(cmd, DLN2_SPI_MODULE_ID)
+
+/* SPI commands */
+#define DLN2_SPI_GET_PORT_COUNT DLN2_SPI_CMD(0x00)
+#define DLN2_SPI_ENABLE DLN2_SPI_CMD(0x11)
+#define DLN2_SPI_DISABLE DLN2_SPI_CMD(0x12)
+#define DLN2_SPI_IS_ENABLED DLN2_SPI_CMD(0x13)
+#define DLN2_SPI_SET_MODE DLN2_SPI_CMD(0x14)
+#define DLN2_SPI_GET_MODE DLN2_SPI_CMD(0x15)
+#define DLN2_SPI_SET_FRAME_SIZE DLN2_SPI_CMD(0x16)
+#define DLN2_SPI_GET_FRAME_SIZE DLN2_SPI_CMD(0x17)
+#define DLN2_SPI_SET_FREQUENCY DLN2_SPI_CMD(0x18)
+#define DLN2_SPI_GET_FREQUENCY DLN2_SPI_CMD(0x19)
+#define DLN2_SPI_READ_WRITE DLN2_SPI_CMD(0x1A)
+#define DLN2_SPI_READ DLN2_SPI_CMD(0x1B)
+#define DLN2_SPI_WRITE DLN2_SPI_CMD(0x1C)
+#define DLN2_SPI_SET_DELAY_BETWEEN_SS DLN2_SPI_CMD(0x20)
+#define DLN2_SPI_GET_DELAY_BETWEEN_SS DLN2_SPI_CMD(0x21)
+#define DLN2_SPI_SET_DELAY_AFTER_SS DLN2_SPI_CMD(0x22)
+#define DLN2_SPI_GET_DELAY_AFTER_SS DLN2_SPI_CMD(0x23)
+#define DLN2_SPI_SET_DELAY_BETWEEN_FRAMES DLN2_SPI_CMD(0x24)
+#define DLN2_SPI_GET_DELAY_BETWEEN_FRAMES DLN2_SPI_CMD(0x25)
+#define DLN2_SPI_SET_SS DLN2_SPI_CMD(0x26)
+#define DLN2_SPI_GET_SS DLN2_SPI_CMD(0x27)
+#define DLN2_SPI_RELEASE_SS DLN2_SPI_CMD(0x28)
+#define DLN2_SPI_SS_VARIABLE_ENABLE DLN2_SPI_CMD(0x2B)
+#define DLN2_SPI_SS_VARIABLE_DISABLE DLN2_SPI_CMD(0x2C)
+#define DLN2_SPI_SS_VARIABLE_IS_ENABLED DLN2_SPI_CMD(0x2D)
+#define DLN2_SPI_SS_AAT_ENABLE DLN2_SPI_CMD(0x2E)
+#define DLN2_SPI_SS_AAT_DISABLE DLN2_SPI_CMD(0x2F)
+#define DLN2_SPI_SS_AAT_IS_ENABLED DLN2_SPI_CMD(0x30)
+#define DLN2_SPI_SS_BETWEEN_FRAMES_ENABLE DLN2_SPI_CMD(0x31)
+#define DLN2_SPI_SS_BETWEEN_FRAMES_DISABLE DLN2_SPI_CMD(0x32)
+#define DLN2_SPI_SS_BETWEEN_FRAMES_IS_ENABLED DLN2_SPI_CMD(0x33)
+#define DLN2_SPI_SET_CPHA DLN2_SPI_CMD(0x34)
+#define DLN2_SPI_GET_CPHA DLN2_SPI_CMD(0x35)
+#define DLN2_SPI_SET_CPOL DLN2_SPI_CMD(0x36)
+#define DLN2_SPI_GET_CPOL DLN2_SPI_CMD(0x37)
+#define DLN2_SPI_SS_MULTI_ENABLE DLN2_SPI_CMD(0x38)
+#define DLN2_SPI_SS_MULTI_DISABLE DLN2_SPI_CMD(0x39)
+#define DLN2_SPI_SS_MULTI_IS_ENABLED DLN2_SPI_CMD(0x3A)
+#define DLN2_SPI_GET_SUPPORTED_MODES DLN2_SPI_CMD(0x40)
+#define DLN2_SPI_GET_SUPPORTED_CPHA_VALUES DLN2_SPI_CMD(0x41)
+#define DLN2_SPI_GET_SUPPORTED_CPOL_VALUES DLN2_SPI_CMD(0x42)
+#define DLN2_SPI_GET_SUPPORTED_FRAME_SIZES DLN2_SPI_CMD(0x43)
+#define DLN2_SPI_GET_SS_COUNT DLN2_SPI_CMD(0x44)
+#define DLN2_SPI_GET_MIN_FREQUENCY DLN2_SPI_CMD(0x45)
+#define DLN2_SPI_GET_MAX_FREQUENCY DLN2_SPI_CMD(0x46)
+#define DLN2_SPI_GET_MIN_DELAY_BETWEEN_SS DLN2_SPI_CMD(0x47)
+#define DLN2_SPI_GET_MAX_DELAY_BETWEEN_SS DLN2_SPI_CMD(0x48)
+#define DLN2_SPI_GET_MIN_DELAY_AFTER_SS DLN2_SPI_CMD(0x49)
+#define DLN2_SPI_GET_MAX_DELAY_AFTER_SS DLN2_SPI_CMD(0x4A)
+#define DLN2_SPI_GET_MIN_DELAY_BETWEEN_FRAMES DLN2_SPI_CMD(0x4B)
+#define DLN2_SPI_GET_MAX_DELAY_BETWEEN_FRAMES DLN2_SPI_CMD(0x4C)
+
+#define DLN2_SPI_MAX_XFER_SIZE 256
+#define DLN2_SPI_BUF_SIZE (DLN2_SPI_MAX_XFER_SIZE + 16)
+#define DLN2_SPI_ATTR_LEAVE_SS_LOW BIT(0)
+#define DLN2_TRANSFERS_WAIT_COMPLETE 1
+#define DLN2_TRANSFERS_CANCEL 0
+#define DLN2_RPM_AUTOSUSPEND_TIMEOUT 2000
+
+struct dln2_spi {
+ struct platform_device *pdev;
+ struct spi_master *master;
+ u8 port;
+
+ /*
+ * This buffer will be used mainly for read/write operations. Since
+ * they're quite large, we cannot use the stack. Protection is not
+ * needed because all SPI communication is serialized by the SPI core.
+ */
+ void *buf;
+
+ u8 bpw;
+ u32 speed;
+ u16 mode;
+ u8 cs;
+};
+
+/*
+ * Enable/Disable SPI module. The disable command will wait for transfers to
+ * complete first.
+ */
+static int dln2_spi_enable(struct dln2_spi *dln2, bool enable)
+{
+ u16 cmd;
+ struct {
+ u8 port;
+ u8 wait_for_completion;
+ } tx;
+ unsigned len = sizeof(tx);
+
+ tx.port = dln2->port;
+
+ if (enable) {
+ cmd = DLN2_SPI_ENABLE;
+ len -= sizeof(tx.wait_for_completion);
+ } else {
+ tx.wait_for_completion = DLN2_TRANSFERS_WAIT_COMPLETE;
+ cmd = DLN2_SPI_DISABLE;
+ }
+
+ return dln2_transfer_tx(dln2->pdev, cmd, &tx, len);
+}
+
+/*
+ * Select/unselect multiple CS lines. The selected lines will be automatically
+ * toggled LOW/HIGH by the board firmware during transfers, provided they're
+ * enabled first.
+ *
+ * Ex: cs_mask = 0x03 -> CS0 & CS1 will be selected and the next WR/RD operation
+ * will toggle the lines LOW/HIGH automatically.
+ */
+static int dln2_spi_cs_set(struct dln2_spi *dln2, u8 cs_mask)
+{
+ struct {
+ u8 port;
+ u8 cs;
+ } tx;
+
+ tx.port = dln2->port;
+
+ /*
+ * According to Diolan docs, "a slave device can be selected by changing
+ * the corresponding bit value to 0". The rest must be set to 1. Hence
+ * the bitwise NOT in front.
+ */
+ tx.cs = ~cs_mask;
+
+ return dln2_transfer_tx(dln2->pdev, DLN2_SPI_SET_SS, &tx, sizeof(tx));
+}
+
+/*
+ * Select one CS line. The other lines will be un-selected.
+ */
+static int dln2_spi_cs_set_one(struct dln2_spi *dln2, u8 cs)
+{
+ return dln2_spi_cs_set(dln2, BIT(cs));
+}
+
+/*
+ * Enable/disable CS lines for usage. The module has to be disabled first.
+ */
+static int dln2_spi_cs_enable(struct dln2_spi *dln2, u8 cs_mask, bool enable)
+{
+ struct {
+ u8 port;
+ u8 cs;
+ } tx;
+ u16 cmd;
+
+ tx.port = dln2->port;
+ tx.cs = cs_mask;
+ cmd = enable ? DLN2_SPI_SS_MULTI_ENABLE : DLN2_SPI_SS_MULTI_DISABLE;
+
+ return dln2_transfer_tx(dln2->pdev, cmd, &tx, sizeof(tx));
+}
+
+static int dln2_spi_cs_enable_all(struct dln2_spi *dln2, bool enable)
+{
+ u8 cs_mask = GENMASK(dln2->master->num_chipselect - 1, 0);
+
+ return dln2_spi_cs_enable(dln2, cs_mask, enable);
+}
+
+static int dln2_spi_get_cs_num(struct dln2_spi *dln2, u16 *cs_num)
+{
+ int ret;
+ struct {
+ u8 port;
+ } tx;
+ struct {
+ __le16 cs_count;
+ } rx;
+ unsigned rx_len = sizeof(rx);
+
+ tx.port = dln2->port;
+ ret = dln2_transfer(dln2->pdev, DLN2_SPI_GET_SS_COUNT, &tx, sizeof(tx),
+ &rx, &rx_len);
+ if (ret < 0)
+ return ret;
+ if (rx_len < sizeof(rx))
+ return -EPROTO;
+
+ *cs_num = le16_to_cpu(rx.cs_count);
+
+ dev_dbg(&dln2->pdev->dev, "cs_num = %d\n", *cs_num);
+
+ return 0;
+}
+
+static int dln2_spi_get_speed(struct dln2_spi *dln2, u16 cmd, u32 *freq)
+{
+ int ret;
+ struct {
+ u8 port;
+ } tx;
+ struct {
+ __le32 speed;
+ } rx;
+ unsigned rx_len = sizeof(rx);
+
+ tx.port = dln2->port;
+
+ ret = dln2_transfer(dln2->pdev, cmd, &tx, sizeof(tx), &rx, &rx_len);
+ if (ret < 0)
+ return ret;
+ if (rx_len < sizeof(rx))
+ return -EPROTO;
+
+ *freq = le32_to_cpu(rx.speed);
+
+ return 0;
+}
+
+/*
+ * Get bus min/max frequencies.
+ */
+static int dln2_spi_get_speed_range(struct dln2_spi *dln2, u32 *fmin, u32 *fmax)
+{
+ int ret;
+
+ ret = dln2_spi_get_speed(dln2, DLN2_SPI_GET_MIN_FREQUENCY, fmin);
+ if (ret < 0)
+ return ret;
+
+ ret = dln2_spi_get_speed(dln2, DLN2_SPI_GET_MAX_FREQUENCY, fmax);
+ if (ret < 0)
+ return ret;
+
+ dev_dbg(&dln2->pdev->dev, "freq_min = %d, freq_max = %d\n",
+ *fmin, *fmax);
+
+ return 0;
+}
+
+/*
+ * Set the bus speed. The module will automatically round down to the closest
+ * available frequency and returns it. The module has to be disabled first.
+ */
+static int dln2_spi_set_speed(struct dln2_spi *dln2, u32 speed)
+{
+ int ret;
+ struct {
+ u8 port;
+ __le32 speed;
+ } __packed tx;
+ struct {
+ __le32 speed;
+ } rx;
+ int rx_len = sizeof(rx);
+
+ tx.port = dln2->port;
+ tx.speed = cpu_to_le32(speed);
+
+ ret = dln2_transfer(dln2->pdev, DLN2_SPI_SET_FREQUENCY, &tx, sizeof(tx),
+ &rx, &rx_len);
+ if (ret < 0)
+ return ret;
+ if (rx_len < sizeof(rx))
+ return -EPROTO;
+
+ return 0;
+}
+
+/*
+ * Change CPOL & CPHA. The module has to be disabled first.
+ */
+static int dln2_spi_set_mode(struct dln2_spi *dln2, u8 mode)
+{
+ struct {
+ u8 port;
+ u8 mode;
+ } tx;
+
+ tx.port = dln2->port;
+ tx.mode = mode;
+
+ return dln2_transfer_tx(dln2->pdev, DLN2_SPI_SET_MODE, &tx, sizeof(tx));
+}
+
+/*
+ * Change frame size. The module has to be disabled first.
+ */
+static int dln2_spi_set_bpw(struct dln2_spi *dln2, u8 bpw)
+{
+ struct {
+ u8 port;
+ u8 bpw;
+ } tx;
+
+ tx.port = dln2->port;
+ tx.bpw = bpw;
+
+ return dln2_transfer_tx(dln2->pdev, DLN2_SPI_SET_FRAME_SIZE,
+ &tx, sizeof(tx));
+}
+
+static int dln2_spi_get_supported_frame_sizes(struct dln2_spi *dln2,
+ u32 *bpw_mask)
+{
+ int ret;
+ struct {
+ u8 port;
+ } tx;
+ struct {
+ u8 count;
+ u8 frame_sizes[36];
+ } *rx = dln2->buf;
+ unsigned rx_len = sizeof(*rx);
+ int i;
+
+ tx.port = dln2->port;
+
+ ret = dln2_transfer(dln2->pdev, DLN2_SPI_GET_SUPPORTED_FRAME_SIZES,
+ &tx, sizeof(tx), rx, &rx_len);
+ if (ret < 0)
+ return ret;
+ if (rx_len < sizeof(*rx))
+ return -EPROTO;
+ if (rx->count > ARRAY_SIZE(rx->frame_sizes))
+ return -EPROTO;
+
+ *bpw_mask = 0;
+ for (i = 0; i < rx->count; i++)
+ *bpw_mask |= BIT(rx->frame_sizes[i] - 1);
+
+ dev_dbg(&dln2->pdev->dev, "bpw_mask = 0x%X\n", *bpw_mask);
+
+ return 0;
+}
+
+/*
+ * Copy the data to DLN2 buffer and change the byte order to LE, requested by
+ * DLN2 module. SPI core makes sure that the data length is a multiple of word
+ * size.
+ */
+static int dln2_spi_copy_to_buf(u8 *dln2_buf, const u8 *src, u16 len, u8 bpw)
+{
+#ifdef __LITTLE_ENDIAN
+ memcpy(dln2_buf, src, len);
+#else
+ if (bpw <= 8) {
+ memcpy(dln2_buf, src, len);
+ } else if (bpw <= 16) {
+ __le16 *d = (__le16 *)dln2_buf;
+ u16 *s = (u16 *)src;
+
+ len = len / 2;
+ while (len--)
+ *d++ = cpu_to_le16p(s++);
+ } else {
+ __le32 *d = (__le32 *)dln2_buf;
+ u32 *s = (u32 *)src;
+
+ len = len / 4;
+ while (len--)
+ *d++ = cpu_to_le32p(s++);
+ }
+#endif
+
+ return 0;
+}
+
+/*
+ * Copy the data from DLN2 buffer and convert to CPU byte order since the DLN2
+ * buffer is LE ordered. SPI core makes sure that the data length is a multiple
+ * of word size. The RX dln2_buf is 2 byte aligned so, for BE, we have to make
+ * sure we avoid unaligned accesses for 32 bit case.
+ */
+static int dln2_spi_copy_from_buf(u8 *dest, const u8 *dln2_buf, u16 len, u8 bpw)
+{
+#ifdef __LITTLE_ENDIAN
+ memcpy(dest, dln2_buf, len);
+#else
+ if (bpw <= 8) {
+ memcpy(dest, dln2_buf, len);
+ } else if (bpw <= 16) {
+ u16 *d = (u16 *)dest;
+ __le16 *s = (__le16 *)dln2_buf;
+
+ len = len / 2;
+ while (len--)
+ *d++ = le16_to_cpup(s++);
+ } else {
+ u32 *d = (u32 *)dest;
+ __le32 *s = (__le32 *)dln2_buf;
+
+ len = len / 4;
+ while (len--)
+ *d++ = get_unaligned_le32(s++);
+ }
+#endif
+
+ return 0;
+}
+
+/*
+ * Perform one write operation.
+ */
+static int dln2_spi_write_one(struct dln2_spi *dln2, const u8 *data,
+ u16 data_len, u8 attr)
+{
+ struct {
+ u8 port;
+ __le16 size;
+ u8 attr;
+ u8 buf[DLN2_SPI_MAX_XFER_SIZE];
+ } __packed *tx = dln2->buf;
+ unsigned tx_len;
+
+ BUILD_BUG_ON(sizeof(*tx) > DLN2_SPI_BUF_SIZE);
+
+ if (data_len > DLN2_SPI_MAX_XFER_SIZE)
+ return -EINVAL;
+
+ tx->port = dln2->port;
+ tx->size = cpu_to_le16(data_len);
+ tx->attr = attr;
+
+ dln2_spi_copy_to_buf(tx->buf, data, data_len, dln2->bpw);
+
+ tx_len = sizeof(*tx) + data_len - DLN2_SPI_MAX_XFER_SIZE;
+ return dln2_transfer_tx(dln2->pdev, DLN2_SPI_WRITE, tx, tx_len);
+}
+
+/*
+ * Perform one read operation.
+ */
+static int dln2_spi_read_one(struct dln2_spi *dln2, u8 *data,
+ u16 data_len, u8 attr)
+{
+ int ret;
+ struct {
+ u8 port;
+ __le16 size;
+ u8 attr;
+ } __packed tx;
+ struct {
+ __le16 size;
+ u8 buf[DLN2_SPI_MAX_XFER_SIZE];
+ } __packed *rx = dln2->buf;
+ unsigned rx_len = sizeof(*rx);
+
+ BUILD_BUG_ON(sizeof(*rx) > DLN2_SPI_BUF_SIZE);
+
+ if (data_len > DLN2_SPI_MAX_XFER_SIZE)
+ return -EINVAL;
+
+ tx.port = dln2->port;
+ tx.size = cpu_to_le16(data_len);
+ tx.attr = attr;
+
+ ret = dln2_transfer(dln2->pdev, DLN2_SPI_READ, &tx, sizeof(tx),
+ rx, &rx_len);
+ if (ret < 0)
+ return ret;
+ if (rx_len < sizeof(rx->size) + data_len)
+ return -EPROTO;
+ if (le16_to_cpu(rx->size) != data_len)
+ return -EPROTO;
+
+ dln2_spi_copy_from_buf(data, rx->buf, data_len, dln2->bpw);
+
+ return 0;
+}
+
+/*
+ * Perform one write & read operation.
+ */
+static int dln2_spi_read_write_one(struct dln2_spi *dln2, const u8 *tx_data,
+ u8 *rx_data, u16 data_len, u8 attr)
+{
+ int ret;
+ struct {
+ u8 port;
+ __le16 size;
+ u8 attr;
+ u8 buf[DLN2_SPI_MAX_XFER_SIZE];
+ } __packed *tx;
+ struct {
+ __le16 size;
+ u8 buf[DLN2_SPI_MAX_XFER_SIZE];
+ } __packed *rx;
+ unsigned tx_len, rx_len;
+
+ BUILD_BUG_ON(sizeof(*tx) > DLN2_SPI_BUF_SIZE ||
+ sizeof(*rx) > DLN2_SPI_BUF_SIZE);
+
+ if (data_len > DLN2_SPI_MAX_XFER_SIZE)
+ return -EINVAL;
+
+ /*
+ * Since this is a pseudo full-duplex communication, we're perfectly
+ * safe to use the same buffer for both tx and rx. When DLN2 sends the
+ * response back, with the rx data, we don't need the tx buffer anymore.
+ */
+ tx = dln2->buf;
+ rx = dln2->buf;
+
+ tx->port = dln2->port;
+ tx->size = cpu_to_le16(data_len);
+ tx->attr = attr;
+
+ dln2_spi_copy_to_buf(tx->buf, tx_data, data_len, dln2->bpw);
+
+ tx_len = sizeof(*tx) + data_len - DLN2_SPI_MAX_XFER_SIZE;
+ rx_len = sizeof(*rx);
+
+ ret = dln2_transfer(dln2->pdev, DLN2_SPI_READ_WRITE, tx, tx_len,
+ rx, &rx_len);
+ if (ret < 0)
+ return ret;
+ if (rx_len < sizeof(rx->size) + data_len)
+ return -EPROTO;
+ if (le16_to_cpu(rx->size) != data_len)
+ return -EPROTO;
+
+ dln2_spi_copy_from_buf(rx_data, rx->buf, data_len, dln2->bpw);
+
+ return 0;
+}
+
+/*
+ * Read/Write wrapper. It will automatically split an operation into multiple
+ * single ones due to device buffer constraints.
+ */
+static int dln2_spi_rdwr(struct dln2_spi *dln2, const u8 *tx_data,
+ u8 *rx_data, u16 data_len, u8 attr) {
+ int ret;
+ u16 len;
+ u8 temp_attr;
+ u16 remaining = data_len;
+ u16 offset;
+
+ do {
+ if (remaining > DLN2_SPI_MAX_XFER_SIZE) {
+ len = DLN2_SPI_MAX_XFER_SIZE;
+ temp_attr = DLN2_SPI_ATTR_LEAVE_SS_LOW;
+ } else {
+ len = remaining;
+ temp_attr = attr;
+ }
+
+ offset = data_len - remaining;
+
+ if (tx_data && rx_data) {
+ ret = dln2_spi_read_write_one(dln2,
+ tx_data + offset,
+ rx_data + offset,
+ len, temp_attr);
+ } else if (tx_data) {
+ ret = dln2_spi_write_one(dln2,
+ tx_data + offset,
+ len, temp_attr);
+ } else if (rx_data) {
+ ret = dln2_spi_read_one(dln2,
+ rx_data + offset,
+ len, temp_attr);
+ } else {
+ return -EINVAL;
+ }
+
+ if (ret < 0)
+ return ret;
+
+ remaining -= len;
+ } while (remaining);
+
+ return 0;
+}
+
+static int dln2_spi_prepare_message(struct spi_master *master,
+ struct spi_message *message)
+{
+ int ret;
+ struct dln2_spi *dln2 = spi_master_get_devdata(master);
+ struct spi_device *spi = message->spi;
+
+ if (dln2->cs != spi->chip_select) {
+ ret = dln2_spi_cs_set_one(dln2, spi->chip_select);
+ if (ret < 0)
+ return ret;
+
+ dln2->cs = spi->chip_select;
+ }
+
+ return 0;
+}
+
+static int dln2_spi_transfer_setup(struct dln2_spi *dln2, u32 speed,
+ u8 bpw, u8 mode)
+{
+ int ret;
+ bool bus_setup_change;
+
+ bus_setup_change = dln2->speed != speed || dln2->mode != mode ||
+ dln2->bpw != bpw;
+
+ if (!bus_setup_change)
+ return 0;
+
+ ret = dln2_spi_enable(dln2, false);
+ if (ret < 0)
+ return ret;
+
+ if (dln2->speed != speed) {
+ ret = dln2_spi_set_speed(dln2, speed);
+ if (ret < 0)
+ return ret;
+
+ dln2->speed = speed;
+ }
+
+ if (dln2->mode != mode) {
+ ret = dln2_spi_set_mode(dln2, mode & 0x3);
+ if (ret < 0)
+ return ret;
+
+ dln2->mode = mode;
+ }
+
+ if (dln2->bpw != bpw) {
+ ret = dln2_spi_set_bpw(dln2, bpw);
+ if (ret < 0)
+ return ret;
+
+ dln2->bpw = bpw;
+ }
+
+ return dln2_spi_enable(dln2, true);
+}
+
+static int dln2_spi_transfer_one(struct spi_master *master,
+ struct spi_device *spi,
+ struct spi_transfer *xfer)
+{
+ struct dln2_spi *dln2 = spi_master_get_devdata(master);
+ int status;
+ u8 attr = 0;
+
+ status = dln2_spi_transfer_setup(dln2, xfer->speed_hz,
+ xfer->bits_per_word,
+ spi->mode);
+ if (status < 0) {
+ dev_err(&dln2->pdev->dev, "Cannot setup transfer\n");
+ return status;
+ }
+
+ if (!xfer->cs_change && !spi_transfer_is_last(master, xfer))
+ attr = DLN2_SPI_ATTR_LEAVE_SS_LOW;
+
+ status = dln2_spi_rdwr(dln2, xfer->tx_buf, xfer->rx_buf,
+ xfer->len, attr);
+ if (status < 0)
+ dev_err(&dln2->pdev->dev, "write/read failed!\n");
+
+ return status;
+}
+
+static int dln2_spi_probe(struct platform_device *pdev)
+{
+ struct spi_master *master;
+ struct dln2_spi *dln2;
+ struct dln2_platform_data *pdata = dev_get_platdata(&pdev->dev);
+ int ret;
+
+ master = spi_alloc_master(&pdev->dev, sizeof(*dln2));
+ if (!master)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, master);
+
+ dln2 = spi_master_get_devdata(master);
+
+ dln2->buf = devm_kmalloc(&pdev->dev, DLN2_SPI_BUF_SIZE, GFP_KERNEL);
+ if (!dln2->buf) {
+ ret = -ENOMEM;
+ goto exit_free_master;
+ }
+
+ dln2->master = master;
+ dln2->pdev = pdev;
+ dln2->port = pdata->port;
+ /* cs/mode can never be 0xff, so the first transfer will set them */
+ dln2->cs = 0xff;
+ dln2->mode = 0xff;
+
+ /* disable SPI module before continuing with the setup */
+ ret = dln2_spi_enable(dln2, false);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "Failed to disable SPI module\n");
+ goto exit_free_master;
+ }
+
+ ret = dln2_spi_get_cs_num(dln2, &master->num_chipselect);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "Failed to get number of CS pins\n");
+ goto exit_free_master;
+ }
+
+ ret = dln2_spi_get_speed_range(dln2,
+ &master->min_speed_hz,
+ &master->max_speed_hz);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "Failed to read bus min/max freqs\n");
+ goto exit_free_master;
+ }
+
+ ret = dln2_spi_get_supported_frame_sizes(dln2,
+ &master->bits_per_word_mask);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "Failed to read supported frame sizes\n");
+ goto exit_free_master;
+ }
+
+ ret = dln2_spi_cs_enable_all(dln2, true);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "Failed to enable CS pins\n");
+ goto exit_free_master;
+ }
+
+ master->bus_num = -1;
+ master->mode_bits = SPI_CPOL | SPI_CPHA;
+ master->prepare_message = dln2_spi_prepare_message;
+ master->transfer_one = dln2_spi_transfer_one;
+ master->auto_runtime_pm = true;
+
+ /* enable SPI module, we're good to go */
+ ret = dln2_spi_enable(dln2, true);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "Failed to enable SPI module\n");
+ goto exit_free_master;
+ }
+
+ pm_runtime_set_autosuspend_delay(&pdev->dev,
+ DLN2_RPM_AUTOSUSPEND_TIMEOUT);
+ pm_runtime_use_autosuspend(&pdev->dev);
+ pm_runtime_set_active(&pdev->dev);
+ pm_runtime_enable(&pdev->dev);
+
+ ret = devm_spi_register_master(&pdev->dev, master);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "Failed to register master\n");
+ goto exit_register;
+ }
+
+ return ret;
+
+exit_register:
+ pm_runtime_disable(&pdev->dev);
+ pm_runtime_set_suspended(&pdev->dev);
+
+ if (dln2_spi_enable(dln2, false) < 0)
+ dev_err(&pdev->dev, "Failed to disable SPI module\n");
+exit_free_master:
+ spi_master_put(master);
+
+ return ret;
+}
+
+static int dln2_spi_remove(struct platform_device *pdev)
+{
+ struct spi_master *master = spi_master_get(platform_get_drvdata(pdev));
+ struct dln2_spi *dln2 = spi_master_get_devdata(master);
+
+ pm_runtime_disable(&pdev->dev);
+
+ if (dln2_spi_enable(dln2, false) < 0)
+ dev_err(&pdev->dev, "Failed to disable SPI module\n");
+
+ return 0;
+}
+
+#ifdef CONFIG_PM_SLEEP
+static int dln2_spi_suspend(struct device *dev)
+{
+ int ret;
+ struct spi_master *master = dev_get_drvdata(dev);
+ struct dln2_spi *dln2 = spi_master_get_devdata(master);
+
+ ret = spi_master_suspend(master);
+ if (ret < 0)
+ return ret;
+
+ if (!pm_runtime_suspended(dev)) {
+ ret = dln2_spi_enable(dln2, false);
+ if (ret < 0)
+ return ret;
+ }
+
+ /*
+ * USB power may be cut off during sleep. Resetting the following
+ * parameters will force the board to be set up before first transfer.
+ */
+ dln2->cs = 0xff;
+ dln2->speed = 0;
+ dln2->bpw = 0;
+ dln2->mode = 0xff;
+
+ return 0;
+}
+
+static int dln2_spi_resume(struct device *dev)
+{
+ int ret;
+ struct spi_master *master = dev_get_drvdata(dev);
+ struct dln2_spi *dln2 = spi_master_get_devdata(master);
+
+ if (!pm_runtime_suspended(dev)) {
+ ret = dln2_spi_cs_enable_all(dln2, true);
+ if (ret < 0)
+ return ret;
+
+ ret = dln2_spi_enable(dln2, true);
+ if (ret < 0)
+ return ret;
+ }
+
+ return spi_master_resume(master);
+}
+#endif /* CONFIG_PM_SLEEP */
+
+#ifdef CONFIG_PM
+static int dln2_spi_runtime_suspend(struct device *dev)
+{
+ struct spi_master *master = dev_get_drvdata(dev);
+ struct dln2_spi *dln2 = spi_master_get_devdata(master);
+
+ return dln2_spi_enable(dln2, false);
+}
+
+static int dln2_spi_runtime_resume(struct device *dev)
+{
+ struct spi_master *master = dev_get_drvdata(dev);
+ struct dln2_spi *dln2 = spi_master_get_devdata(master);
+
+ return dln2_spi_enable(dln2, true);
+}
+#endif /* CONFIG_PM */
+
+static const struct dev_pm_ops dln2_spi_pm = {
+ SET_SYSTEM_SLEEP_PM_OPS(dln2_spi_suspend, dln2_spi_resume)
+ SET_RUNTIME_PM_OPS(dln2_spi_runtime_suspend,
+ dln2_spi_runtime_resume, NULL)
+};
+
+static struct platform_driver spi_dln2_driver = {
+ .driver = {
+ .name = "dln2-spi",
+ .pm = &dln2_spi_pm,
+ },
+ .probe = dln2_spi_probe,
+ .remove = dln2_spi_remove,
+};
+module_platform_driver(spi_dln2_driver);
+
+MODULE_DESCRIPTION("Driver for the Diolan DLN2 SPI master interface");
+MODULE_AUTHOR("Laurentiu Palcu <laurentiu.palcu@intel.com>");
+MODULE_LICENSE("GPL v2");
+MODULE_ALIAS("platform:dln2-spi");
/* Some specific info for SPI0 controller on Intel MID */
-/* HW info for MRST CLk Control Unit, one 32b reg */
+/* HW info for MRST Clk Control Unit, 32b reg per controller */
#define MRST_SPI_CLK_BASE 100000000 /* 100m */
-#define MRST_CLK_SPI0_REG 0xff11d86c
+#define MRST_CLK_SPI_REG 0xff11d86c
#define CLK_SPI_BDIV_OFFSET 0
#define CLK_SPI_BDIV_MASK 0x00000007
#define CLK_SPI_CDIV_OFFSET 9
void __iomem *clk_reg;
u32 clk_cdiv;
- clk_reg = ioremap_nocache(MRST_CLK_SPI0_REG, 16);
+ clk_reg = ioremap_nocache(MRST_CLK_SPI_REG, 16);
if (!clk_reg)
return -ENOMEM;
- /* get SPI controller operating freq info */
- clk_cdiv = (readl(clk_reg) & CLK_SPI_CDIV_MASK) >> CLK_SPI_CDIV_OFFSET;
+ /* Get SPI controller operating freq info */
+ clk_cdiv = readl(clk_reg + dws->bus_num * sizeof(u32));
+ clk_cdiv &= CLK_SPI_CDIV_MASK;
+ clk_cdiv >>= CLK_SPI_CDIV_OFFSET;
dws->max_freq = MRST_SPI_CLK_BASE / (clk_cdiv + 1);
- iounmap(clk_reg);
- dws->num_cs = 16;
+ iounmap(clk_reg);
#ifdef CONFIG_SPI_DW_MID_DMA
dws->dma_priv = kzalloc(sizeof(struct mid_dma), GFP_KERNEL);
struct spi_pci_desc {
int (*setup)(struct dw_spi *);
+ u16 num_cs;
+ u16 bus_num;
};
-static struct spi_pci_desc spi_pci_mid_desc = {
+static struct spi_pci_desc spi_pci_mid_desc_1 = {
.setup = dw_spi_mid_init,
+ .num_cs = 32,
+ .bus_num = 0,
+};
+
+static struct spi_pci_desc spi_pci_mid_desc_2 = {
+ .setup = dw_spi_mid_init,
+ .num_cs = 4,
+ .bus_num = 1,
};
static int spi_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
dws->regs = pcim_iomap_table(pdev)[pci_bar];
- dws->bus_num = 0;
- dws->num_cs = 4;
dws->irq = pdev->irq;
/*
* Specific handling for paltforms, like dma setup,
* clock rate, FIFO depth.
*/
- if (desc && desc->setup) {
- ret = desc->setup(dws);
- if (ret)
- return ret;
+ if (desc) {
+ dws->num_cs = desc->num_cs;
+ dws->bus_num = desc->bus_num;
+
+ if (desc->setup) {
+ ret = desc->setup(dws);
+ if (ret)
+ return ret;
+ }
+ } else {
+ return -ENODEV;
}
ret = dw_spi_add_host(&pdev->dev, dws);
static const struct pci_device_id pci_ids[] = {
/* Intel MID platform SPI controller 0 */
- { PCI_VDEVICE(INTEL, 0x0800), (kernel_ulong_t)&spi_pci_mid_desc},
+ /*
+ * The access to the device 8086:0801 is disabled by HW, since it's
+ * exclusively used by SCU to communicate with MSIC.
+ */
+ /* Intel MID platform SPI controller 1 */
+ { PCI_VDEVICE(INTEL, 0x0800), (kernel_ulong_t)&spi_pci_mid_desc_1},
+ /* Intel MID platform SPI controller 2 */
+ { PCI_VDEVICE(INTEL, 0x0812), (kernel_ulong_t)&spi_pci_mid_desc_2},
{},
};
}
/* Restart the controller, disable all interrupts, clean rx fifo */
-static void spi_hw_init(struct dw_spi *dws)
+static void spi_hw_init(struct device *dev, struct dw_spi *dws)
{
spi_enable_chip(dws, 0);
spi_mask_intr(dws, 0xff);
if (fifo != dw_readw(dws, DW_SPI_TXFLTR))
break;
}
+ dw_writew(dws, DW_SPI_TXFLTR, 0);
dws->fifo_len = (fifo == 2) ? 0 : fifo - 1;
- dw_writew(dws, DW_SPI_TXFLTR, 0);
+ dev_dbg(dev, "Detected FIFO size: %u bytes\n", dws->fifo_len);
}
}
master->dev.of_node = dev->of_node;
/* Basic HW init */
- spi_hw_init(dws);
+ spi_hw_init(dev, dws);
if (dws->dma_ops && dws->dma_ops->dma_init) {
ret = dws->dma_ops->dma_init(dws);
{
int ret;
- spi_hw_init(dws);
+ spi_hw_init(&dws->master->dev, dws);
ret = spi_master_resume(dws->master);
if (ret)
dev_err(&dws->master->dev, "fail to start queue (%d)\n", ret);
return 0;
}
-static int falcon_sflash_prepare_xfer(struct spi_master *master)
-{
- return 0;
-}
-
-static int falcon_sflash_unprepare_xfer(struct spi_master *master)
-{
- return 0;
-}
-
static int falcon_sflash_xfer_one(struct spi_master *master,
struct spi_message *m)
{
master->mode_bits = SPI_MODE_3;
master->flags = SPI_MASTER_HALF_DUPLEX;
master->setup = falcon_sflash_setup;
- master->prepare_transfer_hardware = falcon_sflash_prepare_xfer;
master->transfer_one_message = falcon_sflash_xfer_one;
- master->unprepare_transfer_hardware = falcon_sflash_unprepare_xfer;
master->dev.of_node = pdev->dev.of_node;
ret = devm_spi_register_master(&pdev->dev, master);
#include <linux/dma-mapping.h>
#include <linux/fsl_devices.h>
#include <linux/kernel.h>
+#include <linux/module.h>
#include <linux/of_address.h>
#include <linux/spi/spi.h>
#include <linux/types.h>
}
}
}
+EXPORT_SYMBOL_GPL(fsl_spi_cpm_reinit_txrx);
static void fsl_spi_cpm_bufs_start(struct mpc8xxx_spi *mspi)
{
dma_unmap_single(dev, mspi->tx_dma, t->len, DMA_TO_DEVICE);
return -ENOMEM;
}
+EXPORT_SYMBOL_GPL(fsl_spi_cpm_bufs);
void fsl_spi_cpm_bufs_complete(struct mpc8xxx_spi *mspi)
{
dma_unmap_single(dev, mspi->rx_dma, t->len, DMA_FROM_DEVICE);
mspi->xfer_in_progress = NULL;
}
+EXPORT_SYMBOL_GPL(fsl_spi_cpm_bufs_complete);
void fsl_spi_cpm_irq(struct mpc8xxx_spi *mspi, u32 events)
{
else
complete(&mspi->done);
}
+EXPORT_SYMBOL_GPL(fsl_spi_cpm_irq);
static void *fsl_spi_alloc_dummy_rx(void)
{
fsl_spi_free_dummy_rx();
return -ENOMEM;
}
+EXPORT_SYMBOL_GPL(fsl_spi_cpm_init);
void fsl_spi_cpm_free(struct mpc8xxx_spi *mspi)
{
cpm_muram_free(cpm_muram_offset(mspi->pram));
fsl_spi_free_dummy_rx();
}
+EXPORT_SYMBOL_GPL(fsl_spi_cpm_free);
+
+MODULE_LICENSE("GPL");
};
struct fsl_dspi {
- struct spi_bitbang bitbang;
+ struct spi_master *master;
struct platform_device *pdev;
struct regmap *regmap;
struct clk *clk;
struct spi_transfer *cur_transfer;
+ struct spi_message *cur_msg;
struct chip_data *cur_chip;
size_t len;
void *tx;
char dataflags;
u8 cs;
u16 void_write_data;
+ u32 cs_change;
wait_queue_head_t waitq;
u32 waitflags;
if (dspi->len == 0 || tx_count == DSPI_FIFO_SIZE - 1) {
/* last transfer in the transfer */
dspi_pushr |= SPI_PUSHR_EOQ;
+ if ((dspi->cs_change) && (!dspi->len))
+ dspi_pushr &= ~SPI_PUSHR_CONT;
} else if (tx_word && (dspi->len == 1))
dspi_pushr |= SPI_PUSHR_EOQ;
int rx_count = 0;
int rx_word = is_double_byte_mode(dspi);
u16 d;
+
while ((dspi->rx < dspi->rx_end)
&& (rx_count < DSPI_FIFO_SIZE)) {
if (rx_word) {
return rx_count;
}
-static int dspi_txrx_transfer(struct spi_device *spi, struct spi_transfer *t)
+static int dspi_transfer_one_message(struct spi_master *master,
+ struct spi_message *message)
{
- struct fsl_dspi *dspi = spi_master_get_devdata(spi->master);
- dspi->cur_transfer = t;
- dspi->cur_chip = spi_get_ctldata(spi);
- dspi->cs = spi->chip_select;
- dspi->void_write_data = dspi->cur_chip->void_write_data;
-
- dspi->dataflags = 0;
- dspi->tx = (void *)t->tx_buf;
- dspi->tx_end = dspi->tx + t->len;
- dspi->rx = t->rx_buf;
- dspi->rx_end = dspi->rx + t->len;
- dspi->len = t->len;
-
- if (!dspi->rx)
- dspi->dataflags |= TRAN_STATE_RX_VOID;
-
- if (!dspi->tx)
- dspi->dataflags |= TRAN_STATE_TX_VOID;
-
- regmap_write(dspi->regmap, SPI_MCR, dspi->cur_chip->mcr_val);
- regmap_write(dspi->regmap, SPI_CTAR(dspi->cs), dspi->cur_chip->ctar_val);
- regmap_write(dspi->regmap, SPI_RSER, SPI_RSER_EOQFE);
-
- if (t->speed_hz)
+ struct fsl_dspi *dspi = spi_master_get_devdata(master);
+ struct spi_device *spi = message->spi;
+ struct spi_transfer *transfer;
+ int status = 0;
+ message->actual_length = 0;
+
+ list_for_each_entry(transfer, &message->transfers, transfer_list) {
+ dspi->cur_transfer = transfer;
+ dspi->cur_msg = message;
+ dspi->cur_chip = spi_get_ctldata(spi);
+ dspi->cs = spi->chip_select;
+ if (dspi->cur_transfer->transfer_list.next
+ == &dspi->cur_msg->transfers)
+ transfer->cs_change = 1;
+ dspi->cs_change = transfer->cs_change;
+ dspi->void_write_data = dspi->cur_chip->void_write_data;
+
+ dspi->dataflags = 0;
+ dspi->tx = (void *)transfer->tx_buf;
+ dspi->tx_end = dspi->tx + transfer->len;
+ dspi->rx = transfer->rx_buf;
+ dspi->rx_end = dspi->rx + transfer->len;
+ dspi->len = transfer->len;
+
+ if (!dspi->rx)
+ dspi->dataflags |= TRAN_STATE_RX_VOID;
+
+ if (!dspi->tx)
+ dspi->dataflags |= TRAN_STATE_TX_VOID;
+
+ regmap_write(dspi->regmap, SPI_MCR, dspi->cur_chip->mcr_val);
+ regmap_update_bits(dspi->regmap, SPI_MCR,
+ SPI_MCR_CLR_TXF | SPI_MCR_CLR_RXF,
+ SPI_MCR_CLR_TXF | SPI_MCR_CLR_RXF);
regmap_write(dspi->regmap, SPI_CTAR(dspi->cs),
dspi->cur_chip->ctar_val);
+ if (transfer->speed_hz)
+ regmap_write(dspi->regmap, SPI_CTAR(dspi->cs),
+ dspi->cur_chip->ctar_val);
- dspi_transfer_write(dspi);
+ regmap_write(dspi->regmap, SPI_RSER, SPI_RSER_EOQFE);
+ message->actual_length += dspi_transfer_write(dspi);
- if (wait_event_interruptible(dspi->waitq, dspi->waitflags))
- dev_err(&dspi->pdev->dev, "wait transfer complete fail!\n");
- dspi->waitflags = 0;
-
- return t->len - dspi->len;
-}
+ if (wait_event_interruptible(dspi->waitq, dspi->waitflags))
+ dev_err(&dspi->pdev->dev, "wait transfer complete fail!\n");
+ dspi->waitflags = 0;
-static void dspi_chipselect(struct spi_device *spi, int value)
-{
- struct fsl_dspi *dspi = spi_master_get_devdata(spi->master);
- unsigned int pushr;
-
- regmap_read(dspi->regmap, SPI_PUSHR, &pushr);
-
- switch (value) {
- case BITBANG_CS_ACTIVE:
- pushr |= SPI_PUSHR_CONT;
- break;
- case BITBANG_CS_INACTIVE:
- pushr &= ~SPI_PUSHR_CONT;
- break;
+ if (transfer->delay_usecs)
+ udelay(transfer->delay_usecs);
}
- regmap_write(dspi->regmap, SPI_PUSHR, pushr);
+ message->status = status;
+ spi_finalize_current_message(master);
+
+ return status;
}
-static int dspi_setup_transfer(struct spi_device *spi, struct spi_transfer *t)
+static int dspi_setup(struct spi_device *spi)
{
struct chip_data *chip;
struct fsl_dspi *dspi = spi_master_get_devdata(spi->master);
unsigned char br = 0, pbr = 0, fmsz = 0;
+ if ((spi->bits_per_word >= 4) && (spi->bits_per_word <= 16)) {
+ fmsz = spi->bits_per_word - 1;
+ } else {
+ pr_err("Invalid wordsize\n");
+ return -ENODEV;
+ }
+
/* Only alloc on first setup */
chip = spi_get_ctldata(spi);
if (chip == NULL) {
- chip = devm_kzalloc(&spi->dev, sizeof(struct chip_data),
- GFP_KERNEL);
+ chip = kzalloc(sizeof(struct chip_data), GFP_KERNEL);
if (!chip)
return -ENOMEM;
}
chip->mcr_val = SPI_MCR_MASTER | SPI_MCR_PCSIS |
SPI_MCR_CLR_TXF | SPI_MCR_CLR_RXF;
- if ((spi->bits_per_word >= 4) && (spi->bits_per_word <= 16)) {
- fmsz = spi->bits_per_word - 1;
- } else {
- pr_err("Invalid wordsize\n");
- return -ENODEV;
- }
chip->void_write_data = 0;
return 0;
}
-static int dspi_setup(struct spi_device *spi)
+static void dspi_cleanup(struct spi_device *spi)
{
- if (!spi->max_speed_hz)
- return -EINVAL;
+ struct chip_data *chip = spi_get_ctldata((struct spi_device *)spi);
+
+ dev_dbg(&spi->dev, "spi_device %u.%u cleanup\n",
+ spi->master->bus_num, spi->chip_select);
- return dspi_setup_transfer(spi, NULL);
+ kfree(chip);
}
static irqreturn_t dspi_interrupt(int irq, void *dev_id)
{
struct fsl_dspi *dspi = (struct fsl_dspi *)dev_id;
- regmap_write(dspi->regmap, SPI_SR, SPI_SR_EOQF);
+ struct spi_message *msg = dspi->cur_msg;
+ regmap_write(dspi->regmap, SPI_SR, SPI_SR_EOQF);
dspi_transfer_read(dspi);
if (!dspi->len) {
if (dspi->dataflags & TRAN_STATE_WORD_ODD_NUM)
regmap_update_bits(dspi->regmap, SPI_CTAR(dspi->cs),
- SPI_FRAME_BITS_MASK, SPI_FRAME_BITS(16));
+ SPI_FRAME_BITS_MASK, SPI_FRAME_BITS(16));
dspi->waitflags = 1;
wake_up_interruptible(&dspi->waitq);
- } else {
- dspi_transfer_write(dspi);
-
- return IRQ_HANDLED;
- }
+ } else
+ msg->actual_length += dspi_transfer_write(dspi);
return IRQ_HANDLED;
}
dspi = spi_master_get_devdata(master);
dspi->pdev = pdev;
- dspi->bitbang.master = master;
- dspi->bitbang.chipselect = dspi_chipselect;
- dspi->bitbang.setup_transfer = dspi_setup_transfer;
- dspi->bitbang.txrx_bufs = dspi_txrx_transfer;
- dspi->bitbang.master->setup = dspi_setup;
- dspi->bitbang.master->dev.of_node = pdev->dev.of_node;
+ dspi->master = master;
+
+ master->transfer = NULL;
+ master->setup = dspi_setup;
+ master->transfer_one_message = dspi_transfer_one_message;
+ master->dev.of_node = pdev->dev.of_node;
+ master->cleanup = dspi_cleanup;
master->mode_bits = SPI_CPOL | SPI_CPHA;
master->bits_per_word_mask = SPI_BPW_MASK(4) | SPI_BPW_MASK(8) |
SPI_BPW_MASK(16);
init_waitqueue_head(&dspi->waitq);
platform_set_drvdata(pdev, master);
- ret = spi_bitbang_start(&dspi->bitbang);
+ ret = spi_register_master(master);
if (ret != 0) {
dev_err(&pdev->dev, "Problem registering DSPI master\n");
goto out_clk_put;
struct fsl_dspi *dspi = spi_master_get_devdata(master);
/* Disconnect from the SPI framework */
- spi_bitbang_stop(&dspi->bitbang);
clk_disable_unprepare(dspi->clk);
- spi_master_put(dspi->bitbang.master);
+ spi_unregister_master(dspi->master);
+ spi_master_put(dspi->master);
return 0;
}
#include <linux/interrupt.h>
#include <linux/kernel.h>
#include <linux/mm.h>
+#include <linux/module.h>
#include <linux/of_platform.h>
#include <linux/spi/spi.h>
#ifdef CONFIG_FSL_SOC
type *rx = mpc8xxx_spi->rx; \
*rx++ = (type)(data >> mpc8xxx_spi->rx_shift); \
mpc8xxx_spi->rx = rx; \
-}
+} \
+EXPORT_SYMBOL_GPL(mpc8xxx_spi_rx_buf_##type);
#define MPC8XXX_SPI_TX_BUF(type) \
u32 mpc8xxx_spi_tx_buf_##type(struct mpc8xxx_spi *mpc8xxx_spi) \
data = *tx++ << mpc8xxx_spi->tx_shift; \
mpc8xxx_spi->tx = tx; \
return data; \
-}
+} \
+EXPORT_SYMBOL_GPL(mpc8xxx_spi_tx_buf_##type);
MPC8XXX_SPI_RX_BUF(u8)
MPC8XXX_SPI_RX_BUF(u16)
{
return container_of(pdata, struct mpc8xxx_spi_probe_info, pdata);
}
+EXPORT_SYMBOL_GPL(to_of_pinfo);
const char *mpc8xxx_spi_strmode(unsigned int flags)
{
}
return "CPU";
}
+EXPORT_SYMBOL_GPL(mpc8xxx_spi_strmode);
void mpc8xxx_spi_probe(struct device *dev, struct resource *mem,
unsigned int irq)
mpc8xxx_spi->rx_shift = 0;
mpc8xxx_spi->tx_shift = 0;
- init_completion(&mpc8xxx_spi->done);
-
master->bus_num = pdata->bus_num;
master->num_chipselect = pdata->max_chipselect;
init_completion(&mpc8xxx_spi->done);
}
+EXPORT_SYMBOL_GPL(mpc8xxx_spi_probe);
int mpc8xxx_spi_remove(struct device *dev)
{
return 0;
}
+EXPORT_SYMBOL_GPL(mpc8xxx_spi_remove);
int of_mpc8xxx_spi_probe(struct platform_device *ofdev)
{
return 0;
}
+EXPORT_SYMBOL_GPL(of_mpc8xxx_spi_probe);
+
+MODULE_LICENSE("GPL");
/* rx & tx bufs from the spi_transfer */
const void *tx;
void *rx;
-#ifdef CONFIG_SPI_FSL_ESPI
+#if IS_ENABLED(CONFIG_SPI_FSL_ESPI)
int len;
#endif
unsigned int flags;
-#ifdef CONFIG_SPI_FSL_SPI
+#if IS_ENABLED(CONFIG_SPI_FSL_SPI)
int type;
int native_chipselects;
u8 max_bits_per_word;
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
#include <linux/kernel.h>
#include <linux/module.h>
/*----------------------------------------------------------------------*/
-static inline struct spi_gpio * __pure
+static inline struct spi_gpio *__pure
spi_to_spi_gpio(const struct spi_device *spi)
{
const struct spi_bitbang *bang;
return spi_gpio;
}
-static inline struct spi_gpio_platform_data * __pure
+static inline struct spi_gpio_platform_data *__pure
spi_to_pdata(const struct spi_device *spi)
{
return &spi_to_spi_gpio(spi)->pdata;
unsigned int count = 0;
u32 status;
- while (count < max) {
+ while (count < max / 4) {
spfi_writel(spfi, SPFI_INTERRUPT_SDFUL, SPFI_INTERRUPT_CLEAR);
status = spfi_readl(spfi, SPFI_INTERRUPT_STATUS);
if (status & SPFI_INTERRUPT_SDFUL)
break;
- spfi_writel(spfi, buf[count / 4], SPFI_TX_32BIT_VALID_DATA);
- count += 4;
+ spfi_writel(spfi, buf[count], SPFI_TX_32BIT_VALID_DATA);
+ count++;
}
- return count;
+ return count * 4;
}
static unsigned int spfi_pio_write8(struct img_spfi *spfi, const u8 *buf,
unsigned int count = 0;
u32 status;
- while (count < max) {
+ while (count < max / 4) {
spfi_writel(spfi, SPFI_INTERRUPT_GDEX32BIT,
SPFI_INTERRUPT_CLEAR);
status = spfi_readl(spfi, SPFI_INTERRUPT_STATUS);
if (!(status & SPFI_INTERRUPT_GDEX32BIT))
break;
- buf[count / 4] = spfi_readl(spfi, SPFI_RX_32BIT_VALID_DATA);
- count += 4;
+ buf[count] = spfi_readl(spfi, SPFI_RX_32BIT_VALID_DATA);
+ count++;
}
- return count;
+ return count * 4;
}
static unsigned int spfi_pio_read8(struct img_spfi *spfi, u8 *buf,
time_before(jiffies, timeout)) {
unsigned int tx_count, rx_count;
- switch (xfer->bits_per_word) {
- case 32:
+ if (tx_bytes >= 4)
tx_count = spfi_pio_write32(spfi, tx_buf, tx_bytes);
- rx_count = spfi_pio_read32(spfi, rx_buf, rx_bytes);
- break;
- case 8:
- default:
+ else
tx_count = spfi_pio_write8(spfi, tx_buf, tx_bytes);
+
+ if (rx_bytes >= 4)
+ rx_count = spfi_pio_read32(spfi, rx_buf, rx_bytes);
+ else
rx_count = spfi_pio_read8(spfi, rx_buf, rx_bytes);
- break;
- }
tx_buf += tx_count;
rx_buf += rx_count;
if (xfer->rx_buf) {
rxconf.direction = DMA_DEV_TO_MEM;
- switch (xfer->bits_per_word) {
- case 32:
+ if (xfer->len % 4 == 0) {
rxconf.src_addr = spfi->phys + SPFI_RX_32BIT_VALID_DATA;
rxconf.src_addr_width = 4;
rxconf.src_maxburst = 4;
- break;
- case 8:
- default:
+ } else {
rxconf.src_addr = spfi->phys + SPFI_RX_8BIT_VALID_DATA;
rxconf.src_addr_width = 1;
rxconf.src_maxburst = 4;
if (xfer->tx_buf) {
txconf.direction = DMA_MEM_TO_DEV;
- switch (xfer->bits_per_word) {
- case 32:
+ if (xfer->len % 4 == 0) {
txconf.dst_addr = spfi->phys + SPFI_TX_32BIT_VALID_DATA;
txconf.dst_addr_width = 4;
txconf.dst_maxburst = 4;
- break;
- case 8:
- default:
+ } else {
txconf.dst_addr = spfi->phys + SPFI_TX_8BIT_VALID_DATA;
txconf.dst_addr_width = 1;
txconf.dst_maxburst = 4;
- break;
}
dmaengine_slave_config(spfi->tx_ch, &txconf);
static bool img_spfi_can_dma(struct spi_master *master, struct spi_device *spi,
struct spi_transfer *xfer)
{
- if (xfer->bits_per_word == 8 && xfer->len > SPFI_8BIT_FIFO_SIZE)
- return true;
- if (xfer->bits_per_word == 32 && xfer->len > SPFI_32BIT_FIFO_SIZE)
+ if (xfer->len > SPFI_32BIT_FIFO_SIZE)
return true;
return false;
}
struct completion xfer_done;
void __iomem *base;
- int irq;
struct clk *clk_per;
struct clk *clk_ipg;
unsigned long spi_clk;
struct dma_slave_config slave_config = {};
int ret;
+ /* use pio mode for i.mx6dl chip TKT238285 */
+ if (of_machine_is_compatible("fsl,imx6dl"))
+ return 0;
+
/* Prepare for TX DMA: */
master->dma_tx = dma_request_slave_channel(dev, "tx");
if (!master->dma_tx) {
{
struct dma_async_tx_descriptor *desc_tx = NULL, *desc_rx = NULL;
int ret;
+ unsigned long timeout;
u32 dma;
int left;
struct spi_master *master = spi_imx->bitbang.master;
dma_async_issue_pending(master->dma_tx);
dma_async_issue_pending(master->dma_rx);
/* Wait SDMA to finish the data transfer.*/
- ret = wait_for_completion_timeout(&spi_imx->dma_tx_completion,
+ timeout = wait_for_completion_timeout(&spi_imx->dma_tx_completion,
IMX_DMA_TIMEOUT);
- if (!ret) {
+ if (!timeout) {
pr_warn("%s %s: I/O Error in DMA TX\n",
dev_driver_string(&master->dev),
dev_name(&master->dev));
dmaengine_terminate_all(master->dma_tx);
} else {
- ret = wait_for_completion_timeout(&spi_imx->dma_rx_completion,
- IMX_DMA_TIMEOUT);
- if (!ret) {
+ timeout = wait_for_completion_timeout(
+ &spi_imx->dma_rx_completion, IMX_DMA_TIMEOUT);
+ if (!timeout) {
pr_warn("%s %s: I/O Error in DMA RX\n",
dev_driver_string(&master->dev),
dev_name(&master->dev));
spi_imx->dma_finished = 1;
spi_imx->devtype_data->trigger(spi_imx);
- if (!ret)
+ if (!timeout)
ret = -ETIMEDOUT;
- else if (ret > 0)
+ else
ret = transfer->len;
return ret;
struct spi_master *master;
struct spi_imx_data *spi_imx;
struct resource *res;
- int i, ret, num_cs;
+ int i, ret, num_cs, irq;
if (!np && !mxc_platform_info) {
dev_err(&pdev->dev, "can't get the platform data\n");
goto out_master_put;
}
- spi_imx->irq = platform_get_irq(pdev, 0);
- if (spi_imx->irq < 0) {
- ret = spi_imx->irq;
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0) {
+ ret = irq;
goto out_master_put;
}
- ret = devm_request_irq(&pdev->dev, spi_imx->irq, spi_imx_isr, 0,
+ ret = devm_request_irq(&pdev->dev, irq, spi_imx_isr, 0,
dev_name(&pdev->dev), spi_imx);
if (ret) {
- dev_err(&pdev->dev, "can't get irq%d: %d\n", spi_imx->irq, ret);
+ dev_err(&pdev->dev, "can't get irq%d: %d\n", irq, ret);
goto out_master_put;
}
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
#include <linux/init.h>
struct device *dev;
};
-static struct regmap_config spifc_regmap_config = {
+static const struct regmap_config spifc_regmap_config = {
.reg_bits = 32,
.val_bits = 32,
.reg_stride = 4,
dmaengine_submit(desc);
dma_async_issue_pending(ssp->dmach);
- ret = wait_for_completion_timeout(&spi->c,
- msecs_to_jiffies(SSP_TIMEOUT));
- if (!ret) {
+ if (!wait_for_completion_timeout(&spi->c,
+ msecs_to_jiffies(SSP_TIMEOUT))) {
dev_err(ssp->dev, "DMA transfer timeout\n");
ret = -ETIMEDOUT;
dmaengine_terminate_all(ssp->dmach);
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
*/
#include <linux/kernel.h>
#include <linux/init.h>
* ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
* THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 675 Mass Ave, Cambridge, MA 02139, USA.
*/
#include <linux/kernel.h>
#include <linux/init.h>
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
*/
#include <linux/kernel.h>
/* Runtime PM autosuspend timeout: PM is fairly light on this driver */
#define SPI_AUTOSUSPEND_TIMEOUT 200
-#define ORION_NUM_CHIPSELECTS 1 /* only one slave is supported*/
+/* Some SoCs using this driver support up to 8 chip selects.
+ * It is up to the implementer to only use the chip selects
+ * that are available.
+ */
+#define ORION_NUM_CHIPSELECTS 8
+
#define ORION_SPI_WAIT_RDY_MAX_LOOP 2000 /* in usec */
#define ORION_SPI_IF_CTRL_REG 0x00
#define ARMADA_SPI_CLK_PRESCALE_MASK 0xDF
#define ORION_SPI_MODE_MASK (ORION_SPI_MODE_CPOL | \
ORION_SPI_MODE_CPHA)
+#define ORION_SPI_CS_MASK 0x1C
+#define ORION_SPI_CS_SHIFT 2
+#define ORION_SPI_CS(cs) ((cs << ORION_SPI_CS_SHIFT) & \
+ ORION_SPI_CS_MASK)
enum orion_spi_type {
ORION_SPI,
return 0;
}
-static void orion_spi_set_cs(struct orion_spi *orion_spi, int enable)
+static void orion_spi_set_cs(struct spi_device *spi, bool enable)
{
- if (enable)
+ struct orion_spi *orion_spi;
+
+ orion_spi = spi_master_get_devdata(spi->master);
+
+ orion_spi_clrbits(orion_spi, ORION_SPI_IF_CTRL_REG, ORION_SPI_CS_MASK);
+ orion_spi_setbits(orion_spi, ORION_SPI_IF_CTRL_REG,
+ ORION_SPI_CS(spi->chip_select));
+
+ /* Chip select logic is inverted from spi_set_cs */
+ if (!enable)
orion_spi_setbits(orion_spi, ORION_SPI_IF_CTRL_REG, 0x1);
else
orion_spi_clrbits(orion_spi, ORION_SPI_IF_CTRL_REG, 0x1);
return xfer->len - count;
}
-static int orion_spi_transfer_one_message(struct spi_master *master,
- struct spi_message *m)
+static int orion_spi_transfer_one(struct spi_master *master,
+ struct spi_device *spi,
+ struct spi_transfer *t)
{
- struct orion_spi *orion_spi = spi_master_get_devdata(master);
- struct spi_device *spi = m->spi;
- struct spi_transfer *t = NULL;
- int par_override = 0;
int status = 0;
- int cs_active = 0;
-
- /* Load defaults */
- status = orion_spi_setup_transfer(spi, NULL);
+ status = orion_spi_setup_transfer(spi, t);
if (status < 0)
- goto msg_done;
-
- list_for_each_entry(t, &m->transfers, transfer_list) {
- if (par_override || t->speed_hz || t->bits_per_word) {
- par_override = 1;
- status = orion_spi_setup_transfer(spi, t);
- if (status < 0)
- break;
- if (!t->speed_hz && !t->bits_per_word)
- par_override = 0;
- }
-
- if (!cs_active) {
- orion_spi_set_cs(orion_spi, 1);
- cs_active = 1;
- }
+ return status;
- if (t->len)
- m->actual_length += orion_spi_write_read(spi, t);
+ if (t->len)
+ orion_spi_write_read(spi, t);
- if (t->delay_usecs)
- udelay(t->delay_usecs);
-
- if (t->cs_change) {
- orion_spi_set_cs(orion_spi, 0);
- cs_active = 0;
- }
- }
-
-msg_done:
- if (cs_active)
- orion_spi_set_cs(orion_spi, 0);
-
- m->status = status;
- spi_finalize_current_message(master);
+ return status;
+}
- return 0;
+static int orion_spi_setup(struct spi_device *spi)
+{
+ return orion_spi_setup_transfer(spi, NULL);
}
static int orion_spi_reset(struct orion_spi *orion_spi)
{
/* Verify that the CS is deasserted */
- orion_spi_set_cs(orion_spi, 0);
-
+ orion_spi_clrbits(orion_spi, ORION_SPI_IF_CTRL_REG, 0x1);
return 0;
}
/* we support only mode 0, and no options */
master->mode_bits = SPI_CPHA | SPI_CPOL;
-
- master->transfer_one_message = orion_spi_transfer_one_message;
+ master->set_cs = orion_spi_set_cs;
+ master->transfer_one = orion_spi_transfer_one;
master->num_chipselect = ORION_NUM_CHIPSELECTS;
+ master->setup = orion_spi_setup;
master->bits_per_word_mask = SPI_BPW_MASK(8) | SPI_BPW_MASK(16);
master->auto_runtime_pm = true;
* by using ->dma_running.
*/
if (atomic_dec_and_test(&drv_data->dma_running)) {
- void __iomem *reg = drv_data->ioaddr;
-
/*
* If the other CPU is still handling the ROR interrupt we
* might not know about the error yet. So we re-check the
* ROR bit here before we clear the status register.
*/
if (!error) {
- u32 status = read_SSSR(reg) & drv_data->mask_sr;
+ u32 status = pxa2xx_spi_read(drv_data, SSSR)
+ & drv_data->mask_sr;
error = status & SSSR_ROR;
}
/* Clear status & disable interrupts */
- write_SSCR1(read_SSCR1(reg) & ~drv_data->dma_cr1, reg);
+ pxa2xx_spi_write(drv_data, SSCR1,
+ pxa2xx_spi_read(drv_data, SSCR1)
+ & ~drv_data->dma_cr1);
write_SSSR_CS(drv_data, drv_data->clear_sr);
if (!pxa25x_ssp_comp(drv_data))
- write_SSTO(0, reg);
+ pxa2xx_spi_write(drv_data, SSTO, 0);
if (!error) {
pxa2xx_spi_unmap_dma_buffers(drv_data);
msg->state = pxa2xx_spi_next_transfer(drv_data);
} else {
/* In case we got an error we disable the SSP now */
- write_SSCR0(read_SSCR0(reg) & ~SSCR0_SSE, reg);
+ pxa2xx_spi_write(drv_data, SSCR0,
+ pxa2xx_spi_read(drv_data, SSCR0)
+ & ~SSCR0_SSE);
msg->state = ERROR_STATE;
}
{
u32 status;
- status = read_SSSR(drv_data->ioaddr) & drv_data->mask_sr;
+ status = pxa2xx_spi_read(drv_data, SSSR) & drv_data->mask_sr;
if (status & SSSR_ROR) {
dev_err(&drv_data->pdev->dev, "FIFO overrun\n");
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
#include <linux/delay.h>
#include <linux/spi/spi.h>
#include <linux/spi/pxa2xx_spi.h>
+#include <mach/dma.h>
#include "spi-pxa2xx.h"
#define DMA_INT_MASK (DCSR_ENDINTR | DCSR_STARTINTR | DCSR_BUSERR)
drv_data->dma_mapped = 0;
}
-static int wait_ssp_rx_stall(void const __iomem *ioaddr)
+static int wait_ssp_rx_stall(struct driver_data *drv_data)
{
unsigned long limit = loops_per_jiffy << 1;
- while ((read_SSSR(ioaddr) & SSSR_BSY) && --limit)
+ while ((pxa2xx_spi_read(drv_data, SSSR) & SSSR_BSY) && --limit)
cpu_relax();
return limit;
static void pxa2xx_spi_dma_error_stop(struct driver_data *drv_data,
const char *msg)
{
- void __iomem *reg = drv_data->ioaddr;
-
/* Stop and reset */
DCSR(drv_data->rx_channel) = RESET_DMA_CHANNEL;
DCSR(drv_data->tx_channel) = RESET_DMA_CHANNEL;
write_SSSR_CS(drv_data, drv_data->clear_sr);
- write_SSCR1(read_SSCR1(reg) & ~drv_data->dma_cr1, reg);
+ pxa2xx_spi_write(drv_data, SSCR1,
+ pxa2xx_spi_read(drv_data, SSCR1)
+ & ~drv_data->dma_cr1);
if (!pxa25x_ssp_comp(drv_data))
- write_SSTO(0, reg);
+ pxa2xx_spi_write(drv_data, SSTO, 0);
pxa2xx_spi_flush(drv_data);
- write_SSCR0(read_SSCR0(reg) & ~SSCR0_SSE, reg);
+ pxa2xx_spi_write(drv_data, SSCR0,
+ pxa2xx_spi_read(drv_data, SSCR0) & ~SSCR0_SSE);
pxa2xx_spi_unmap_dma_buffers(drv_data);
static void pxa2xx_spi_dma_transfer_complete(struct driver_data *drv_data)
{
- void __iomem *reg = drv_data->ioaddr;
struct spi_message *msg = drv_data->cur_msg;
/* Clear and disable interrupts on SSP and DMA channels*/
- write_SSCR1(read_SSCR1(reg) & ~drv_data->dma_cr1, reg);
+ pxa2xx_spi_write(drv_data, SSCR1,
+ pxa2xx_spi_read(drv_data, SSCR1)
+ & ~drv_data->dma_cr1);
write_SSSR_CS(drv_data, drv_data->clear_sr);
DCSR(drv_data->tx_channel) = RESET_DMA_CHANNEL;
DCSR(drv_data->rx_channel) = RESET_DMA_CHANNEL;
&& (drv_data->ssp_type == PXA25x_SSP)) {
/* Wait for rx to stall */
- if (wait_ssp_rx_stall(drv_data->ioaddr) == 0)
+ if (wait_ssp_rx_stall(drv_data) == 0)
dev_err(&drv_data->pdev->dev,
"dma_handler: ssp rx stall failed\n");
irqreturn_t pxa2xx_spi_dma_transfer(struct driver_data *drv_data)
{
u32 irq_status;
- void __iomem *reg = drv_data->ioaddr;
- irq_status = read_SSSR(reg) & drv_data->mask_sr;
+ irq_status = pxa2xx_spi_read(drv_data, SSSR) & drv_data->mask_sr;
if (irq_status & SSSR_ROR) {
pxa2xx_spi_dma_error_stop(drv_data,
"dma_transfer: fifo overrun");
/* Check for false positive timeout */
if ((irq_status & SSSR_TINT)
&& (DCSR(drv_data->tx_channel) & DCSR_RUN)) {
- write_SSSR(SSSR_TINT, reg);
+ pxa2xx_spi_write(drv_data, SSSR, SSSR_TINT);
return IRQ_HANDLED;
}
/* Clear and disable timeout interrupt, do the rest in
* dma_transfer_complete */
if (!pxa25x_ssp_comp(drv_data))
- write_SSTO(0, reg);
+ pxa2xx_spi_write(drv_data, SSTO, 0);
/* finish this transfer, start the next */
pxa2xx_spi_dma_transfer_complete(drv_data);
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
#include <linux/init.h>
MODULE_LICENSE("GPL");
MODULE_ALIAS("platform:pxa2xx-spi");
-#define MAX_BUSES 3
-
#define TIMOUT_DFLT 1000
/*
static bool pxa2xx_spi_txfifo_full(const struct driver_data *drv_data)
{
- void __iomem *reg = drv_data->ioaddr;
u32 mask;
switch (drv_data->ssp_type) {
break;
}
- return (read_SSSR(reg) & mask) == mask;
+ return (pxa2xx_spi_read(drv_data, SSSR) & mask) == mask;
}
static void pxa2xx_spi_clear_rx_thre(const struct driver_data *drv_data,
unsigned offset = 0x400;
u32 value, orig;
- if (!is_lpss_ssp(drv_data))
- return;
-
/*
* Perform auto-detection of the LPSS SSP private registers. They
* can be either at 1k or 2k offset from the base address.
{
u32 value;
- if (!is_lpss_ssp(drv_data))
- return;
-
value = __lpss_ssp_read_priv(drv_data, SPI_CS_CONTROL);
if (enable)
value &= ~SPI_CS_CONTROL_CS_HIGH;
struct chip_data *chip = drv_data->cur_chip;
if (drv_data->ssp_type == CE4100_SSP) {
- write_SSSR(drv_data->cur_chip->frm, drv_data->ioaddr);
+ pxa2xx_spi_write(drv_data, SSSR, drv_data->cur_chip->frm);
return;
}
return;
}
- lpss_ssp_cs_control(drv_data, true);
+ if (is_lpss_ssp(drv_data))
+ lpss_ssp_cs_control(drv_data, true);
}
static void cs_deassert(struct driver_data *drv_data)
return;
}
- lpss_ssp_cs_control(drv_data, false);
+ if (is_lpss_ssp(drv_data))
+ lpss_ssp_cs_control(drv_data, false);
}
int pxa2xx_spi_flush(struct driver_data *drv_data)
{
unsigned long limit = loops_per_jiffy << 1;
- void __iomem *reg = drv_data->ioaddr;
-
do {
- while (read_SSSR(reg) & SSSR_RNE) {
- read_SSDR(reg);
- }
- } while ((read_SSSR(reg) & SSSR_BSY) && --limit);
+ while (pxa2xx_spi_read(drv_data, SSSR) & SSSR_RNE)
+ pxa2xx_spi_read(drv_data, SSDR);
+ } while ((pxa2xx_spi_read(drv_data, SSSR) & SSSR_BSY) && --limit);
write_SSSR_CS(drv_data, SSSR_ROR);
return limit;
static int null_writer(struct driver_data *drv_data)
{
- void __iomem *reg = drv_data->ioaddr;
u8 n_bytes = drv_data->n_bytes;
if (pxa2xx_spi_txfifo_full(drv_data)
|| (drv_data->tx == drv_data->tx_end))
return 0;
- write_SSDR(0, reg);
+ pxa2xx_spi_write(drv_data, SSDR, 0);
drv_data->tx += n_bytes;
return 1;
static int null_reader(struct driver_data *drv_data)
{
- void __iomem *reg = drv_data->ioaddr;
u8 n_bytes = drv_data->n_bytes;
- while ((read_SSSR(reg) & SSSR_RNE)
- && (drv_data->rx < drv_data->rx_end)) {
- read_SSDR(reg);
+ while ((pxa2xx_spi_read(drv_data, SSSR) & SSSR_RNE)
+ && (drv_data->rx < drv_data->rx_end)) {
+ pxa2xx_spi_read(drv_data, SSDR);
drv_data->rx += n_bytes;
}
static int u8_writer(struct driver_data *drv_data)
{
- void __iomem *reg = drv_data->ioaddr;
-
if (pxa2xx_spi_txfifo_full(drv_data)
|| (drv_data->tx == drv_data->tx_end))
return 0;
- write_SSDR(*(u8 *)(drv_data->tx), reg);
+ pxa2xx_spi_write(drv_data, SSDR, *(u8 *)(drv_data->tx));
++drv_data->tx;
return 1;
static int u8_reader(struct driver_data *drv_data)
{
- void __iomem *reg = drv_data->ioaddr;
-
- while ((read_SSSR(reg) & SSSR_RNE)
- && (drv_data->rx < drv_data->rx_end)) {
- *(u8 *)(drv_data->rx) = read_SSDR(reg);
+ while ((pxa2xx_spi_read(drv_data, SSSR) & SSSR_RNE)
+ && (drv_data->rx < drv_data->rx_end)) {
+ *(u8 *)(drv_data->rx) = pxa2xx_spi_read(drv_data, SSDR);
++drv_data->rx;
}
static int u16_writer(struct driver_data *drv_data)
{
- void __iomem *reg = drv_data->ioaddr;
-
if (pxa2xx_spi_txfifo_full(drv_data)
|| (drv_data->tx == drv_data->tx_end))
return 0;
- write_SSDR(*(u16 *)(drv_data->tx), reg);
+ pxa2xx_spi_write(drv_data, SSDR, *(u16 *)(drv_data->tx));
drv_data->tx += 2;
return 1;
static int u16_reader(struct driver_data *drv_data)
{
- void __iomem *reg = drv_data->ioaddr;
-
- while ((read_SSSR(reg) & SSSR_RNE)
- && (drv_data->rx < drv_data->rx_end)) {
- *(u16 *)(drv_data->rx) = read_SSDR(reg);
+ while ((pxa2xx_spi_read(drv_data, SSSR) & SSSR_RNE)
+ && (drv_data->rx < drv_data->rx_end)) {
+ *(u16 *)(drv_data->rx) = pxa2xx_spi_read(drv_data, SSDR);
drv_data->rx += 2;
}
static int u32_writer(struct driver_data *drv_data)
{
- void __iomem *reg = drv_data->ioaddr;
-
if (pxa2xx_spi_txfifo_full(drv_data)
|| (drv_data->tx == drv_data->tx_end))
return 0;
- write_SSDR(*(u32 *)(drv_data->tx), reg);
+ pxa2xx_spi_write(drv_data, SSDR, *(u32 *)(drv_data->tx));
drv_data->tx += 4;
return 1;
static int u32_reader(struct driver_data *drv_data)
{
- void __iomem *reg = drv_data->ioaddr;
-
- while ((read_SSSR(reg) & SSSR_RNE)
- && (drv_data->rx < drv_data->rx_end)) {
- *(u32 *)(drv_data->rx) = read_SSDR(reg);
+ while ((pxa2xx_spi_read(drv_data, SSSR) & SSSR_RNE)
+ && (drv_data->rx < drv_data->rx_end)) {
+ *(u32 *)(drv_data->rx) = pxa2xx_spi_read(drv_data, SSDR);
drv_data->rx += 4;
}
static void reset_sccr1(struct driver_data *drv_data)
{
- void __iomem *reg = drv_data->ioaddr;
struct chip_data *chip = drv_data->cur_chip;
u32 sccr1_reg;
- sccr1_reg = read_SSCR1(reg) & ~drv_data->int_cr1;
+ sccr1_reg = pxa2xx_spi_read(drv_data, SSCR1) & ~drv_data->int_cr1;
sccr1_reg &= ~SSCR1_RFT;
sccr1_reg |= chip->threshold;
- write_SSCR1(sccr1_reg, reg);
+ pxa2xx_spi_write(drv_data, SSCR1, sccr1_reg);
}
static void int_error_stop(struct driver_data *drv_data, const char* msg)
{
- void __iomem *reg = drv_data->ioaddr;
-
/* Stop and reset SSP */
write_SSSR_CS(drv_data, drv_data->clear_sr);
reset_sccr1(drv_data);
if (!pxa25x_ssp_comp(drv_data))
- write_SSTO(0, reg);
+ pxa2xx_spi_write(drv_data, SSTO, 0);
pxa2xx_spi_flush(drv_data);
- write_SSCR0(read_SSCR0(reg) & ~SSCR0_SSE, reg);
+ pxa2xx_spi_write(drv_data, SSCR0,
+ pxa2xx_spi_read(drv_data, SSCR0) & ~SSCR0_SSE);
dev_err(&drv_data->pdev->dev, "%s\n", msg);
static void int_transfer_complete(struct driver_data *drv_data)
{
- void __iomem *reg = drv_data->ioaddr;
-
/* Stop SSP */
write_SSSR_CS(drv_data, drv_data->clear_sr);
reset_sccr1(drv_data);
if (!pxa25x_ssp_comp(drv_data))
- write_SSTO(0, reg);
+ pxa2xx_spi_write(drv_data, SSTO, 0);
/* Update total byte transferred return count actual bytes read */
drv_data->cur_msg->actual_length += drv_data->len -
static irqreturn_t interrupt_transfer(struct driver_data *drv_data)
{
- void __iomem *reg = drv_data->ioaddr;
+ u32 irq_mask = (pxa2xx_spi_read(drv_data, SSCR1) & SSCR1_TIE) ?
+ drv_data->mask_sr : drv_data->mask_sr & ~SSSR_TFS;
- u32 irq_mask = (read_SSCR1(reg) & SSCR1_TIE) ?
- drv_data->mask_sr : drv_data->mask_sr & ~SSSR_TFS;
-
- u32 irq_status = read_SSSR(reg) & irq_mask;
+ u32 irq_status = pxa2xx_spi_read(drv_data, SSSR) & irq_mask;
if (irq_status & SSSR_ROR) {
int_error_stop(drv_data, "interrupt_transfer: fifo overrun");
}
if (irq_status & SSSR_TINT) {
- write_SSSR(SSSR_TINT, reg);
+ pxa2xx_spi_write(drv_data, SSSR, SSSR_TINT);
if (drv_data->read(drv_data)) {
int_transfer_complete(drv_data);
return IRQ_HANDLED;
u32 bytes_left;
u32 sccr1_reg;
- sccr1_reg = read_SSCR1(reg);
+ sccr1_reg = pxa2xx_spi_read(drv_data, SSCR1);
sccr1_reg &= ~SSCR1_TIE;
/*
pxa2xx_spi_set_rx_thre(drv_data, &sccr1_reg, rx_thre);
}
- write_SSCR1(sccr1_reg, reg);
+ pxa2xx_spi_write(drv_data, SSCR1, sccr1_reg);
}
/* We did something */
static irqreturn_t ssp_int(int irq, void *dev_id)
{
struct driver_data *drv_data = dev_id;
- void __iomem *reg = drv_data->ioaddr;
u32 sccr1_reg;
u32 mask = drv_data->mask_sr;
u32 status;
* are all set to one. That means that the device is already
* powered off.
*/
- status = read_SSSR(reg);
+ status = pxa2xx_spi_read(drv_data, SSSR);
if (status == ~0)
return IRQ_NONE;
- sccr1_reg = read_SSCR1(reg);
+ sccr1_reg = pxa2xx_spi_read(drv_data, SSCR1);
/* Ignore possible writes if we don't need to write */
if (!(sccr1_reg & SSCR1_TIE))
if (!drv_data->cur_msg) {
- write_SSCR0(read_SSCR0(reg) & ~SSCR0_SSE, reg);
- write_SSCR1(read_SSCR1(reg) & ~drv_data->int_cr1, reg);
+ pxa2xx_spi_write(drv_data, SSCR0,
+ pxa2xx_spi_read(drv_data, SSCR0)
+ & ~SSCR0_SSE);
+ pxa2xx_spi_write(drv_data, SSCR1,
+ pxa2xx_spi_read(drv_data, SSCR1)
+ & ~drv_data->int_cr1);
if (!pxa25x_ssp_comp(drv_data))
- write_SSTO(0, reg);
+ pxa2xx_spi_write(drv_data, SSTO, 0);
write_SSSR_CS(drv_data, drv_data->clear_sr);
dev_err(&drv_data->pdev->dev,
struct spi_transfer *transfer = NULL;
struct spi_transfer *previous = NULL;
struct chip_data *chip = NULL;
- void __iomem *reg = drv_data->ioaddr;
u32 clk_div = 0;
u8 bits = 0;
u32 speed = 0;
/* Clear status and start DMA engine */
cr1 = chip->cr1 | dma_thresh | drv_data->dma_cr1;
- write_SSSR(drv_data->clear_sr, reg);
+ pxa2xx_spi_write(drv_data, SSSR, drv_data->clear_sr);
pxa2xx_spi_dma_start(drv_data);
} else {
}
if (is_lpss_ssp(drv_data)) {
- if ((read_SSIRF(reg) & 0xff) != chip->lpss_rx_threshold)
- write_SSIRF(chip->lpss_rx_threshold, reg);
- if ((read_SSITF(reg) & 0xffff) != chip->lpss_tx_threshold)
- write_SSITF(chip->lpss_tx_threshold, reg);
+ if ((pxa2xx_spi_read(drv_data, SSIRF) & 0xff)
+ != chip->lpss_rx_threshold)
+ pxa2xx_spi_write(drv_data, SSIRF,
+ chip->lpss_rx_threshold);
+ if ((pxa2xx_spi_read(drv_data, SSITF) & 0xffff)
+ != chip->lpss_tx_threshold)
+ pxa2xx_spi_write(drv_data, SSITF,
+ chip->lpss_tx_threshold);
}
if (is_quark_x1000_ssp(drv_data) &&
- (read_DDS_RATE(reg) != chip->dds_rate))
- write_DDS_RATE(chip->dds_rate, reg);
+ (pxa2xx_spi_read(drv_data, DDS_RATE) != chip->dds_rate))
+ pxa2xx_spi_write(drv_data, DDS_RATE, chip->dds_rate);
/* see if we need to reload the config registers */
- if ((read_SSCR0(reg) != cr0) ||
- (read_SSCR1(reg) & change_mask) != (cr1 & change_mask)) {
-
+ if ((pxa2xx_spi_read(drv_data, SSCR0) != cr0)
+ || (pxa2xx_spi_read(drv_data, SSCR1) & change_mask)
+ != (cr1 & change_mask)) {
/* stop the SSP, and update the other bits */
- write_SSCR0(cr0 & ~SSCR0_SSE, reg);
+ pxa2xx_spi_write(drv_data, SSCR0, cr0 & ~SSCR0_SSE);
if (!pxa25x_ssp_comp(drv_data))
- write_SSTO(chip->timeout, reg);
+ pxa2xx_spi_write(drv_data, SSTO, chip->timeout);
/* first set CR1 without interrupt and service enables */
- write_SSCR1(cr1 & change_mask, reg);
+ pxa2xx_spi_write(drv_data, SSCR1, cr1 & change_mask);
/* restart the SSP */
- write_SSCR0(cr0, reg);
+ pxa2xx_spi_write(drv_data, SSCR0, cr0);
} else {
if (!pxa25x_ssp_comp(drv_data))
- write_SSTO(chip->timeout, reg);
+ pxa2xx_spi_write(drv_data, SSTO, chip->timeout);
}
cs_assert(drv_data);
/* after chip select, release the data by enabling service
* requests and interrupts, without changing any mode bits */
- write_SSCR1(cr1, reg);
+ pxa2xx_spi_write(drv_data, SSCR1, cr1);
}
static int pxa2xx_spi_transfer_one_message(struct spi_master *master,
struct driver_data *drv_data = spi_master_get_devdata(master);
/* Disable the SSP now */
- write_SSCR0(read_SSCR0(drv_data->ioaddr) & ~SSCR0_SSE,
- drv_data->ioaddr);
+ pxa2xx_spi_write(drv_data, SSCR0,
+ pxa2xx_spi_read(drv_data, SSCR0) & ~SSCR0_SSE);
return 0;
}
struct driver_data *drv_data;
struct ssp_device *ssp;
int status;
+ u32 tmp;
platform_info = dev_get_platdata(dev);
if (!platform_info) {
drv_data->max_clk_rate = clk_get_rate(ssp->clk);
/* Load default SSP configuration */
- write_SSCR0(0, drv_data->ioaddr);
+ pxa2xx_spi_write(drv_data, SSCR0, 0);
switch (drv_data->ssp_type) {
case QUARK_X1000_SSP:
- write_SSCR1(QUARK_X1000_SSCR1_RxTresh(
- RX_THRESH_QUARK_X1000_DFLT) |
- QUARK_X1000_SSCR1_TxTresh(
- TX_THRESH_QUARK_X1000_DFLT),
- drv_data->ioaddr);
+ tmp = QUARK_X1000_SSCR1_RxTresh(RX_THRESH_QUARK_X1000_DFLT)
+ | QUARK_X1000_SSCR1_TxTresh(TX_THRESH_QUARK_X1000_DFLT);
+ pxa2xx_spi_write(drv_data, SSCR1, tmp);
/* using the Motorola SPI protocol and use 8 bit frame */
- write_SSCR0(QUARK_X1000_SSCR0_Motorola
- | QUARK_X1000_SSCR0_DataSize(8),
- drv_data->ioaddr);
+ pxa2xx_spi_write(drv_data, SSCR0,
+ QUARK_X1000_SSCR0_Motorola
+ | QUARK_X1000_SSCR0_DataSize(8));
break;
default:
- write_SSCR1(SSCR1_RxTresh(RX_THRESH_DFLT) |
- SSCR1_TxTresh(TX_THRESH_DFLT),
- drv_data->ioaddr);
- write_SSCR0(SSCR0_SCR(2)
- | SSCR0_Motorola
- | SSCR0_DataSize(8),
- drv_data->ioaddr);
+ tmp = SSCR1_RxTresh(RX_THRESH_DFLT) |
+ SSCR1_TxTresh(TX_THRESH_DFLT);
+ pxa2xx_spi_write(drv_data, SSCR1, tmp);
+ tmp = SSCR0_SCR(2) | SSCR0_Motorola | SSCR0_DataSize(8);
+ pxa2xx_spi_write(drv_data, SSCR0, tmp);
break;
}
if (!pxa25x_ssp_comp(drv_data))
- write_SSTO(0, drv_data->ioaddr);
+ pxa2xx_spi_write(drv_data, SSTO, 0);
if (!is_quark_x1000_ssp(drv_data))
- write_SSPSP(0, drv_data->ioaddr);
+ pxa2xx_spi_write(drv_data, SSPSP, 0);
- lpss_ssp_setup(drv_data);
+ if (is_lpss_ssp(drv_data))
+ lpss_ssp_setup(drv_data);
tasklet_init(&drv_data->pump_transfers, pump_transfers,
(unsigned long)drv_data);
pm_runtime_get_sync(&pdev->dev);
/* Disable the SSP at the peripheral and SOC level */
- write_SSCR0(0, drv_data->ioaddr);
+ pxa2xx_spi_write(drv_data, SSCR0, 0);
clk_disable_unprepare(ssp->clk);
/* Release DMA */
status = spi_master_suspend(drv_data->master);
if (status != 0)
return status;
- write_SSCR0(0, drv_data->ioaddr);
+ pxa2xx_spi_write(drv_data, SSCR0, 0);
if (!pm_runtime_suspended(dev))
clk_disable_unprepare(ssp->clk);
clk_prepare_enable(ssp->clk);
/* Restore LPSS private register bits */
- lpss_ssp_setup(drv_data);
+ if (is_lpss_ssp(drv_data))
+ lpss_ssp_setup(drv_data);
/* Start the queue running */
status = spi_master_resume(drv_data->master);
void (*cs_control)(u32 command);
};
-#define DEFINE_SSP_REG(reg, off) \
-static inline u32 read_##reg(void const __iomem *p) \
-{ return __raw_readl(p + (off)); } \
-\
-static inline void write_##reg(u32 v, void __iomem *p) \
-{ __raw_writel(v, p + (off)); }
-
-DEFINE_SSP_REG(SSCR0, 0x00)
-DEFINE_SSP_REG(SSCR1, 0x04)
-DEFINE_SSP_REG(SSSR, 0x08)
-DEFINE_SSP_REG(SSITR, 0x0c)
-DEFINE_SSP_REG(SSDR, 0x10)
-DEFINE_SSP_REG(DDS_RATE, 0x28) /* DDS Clock Rate */
-DEFINE_SSP_REG(SSTO, 0x28)
-DEFINE_SSP_REG(SSPSP, 0x2c)
-DEFINE_SSP_REG(SSITF, SSITF)
-DEFINE_SSP_REG(SSIRF, SSIRF)
+static inline u32 pxa2xx_spi_read(const struct driver_data *drv_data,
+ unsigned reg)
+{
+ return __raw_readl(drv_data->ioaddr + reg);
+}
+
+static inline void pxa2xx_spi_write(const struct driver_data *drv_data,
+ unsigned reg, u32 val)
+{
+ __raw_writel(val, drv_data->ioaddr + reg);
+}
#define START_STATE ((void *)0)
#define RUNNING_STATE ((void *)1)
static inline void write_SSSR_CS(struct driver_data *drv_data, u32 val)
{
- void __iomem *reg = drv_data->ioaddr;
-
if (drv_data->ssp_type == CE4100_SSP ||
drv_data->ssp_type == QUARK_X1000_SSP)
- val |= read_SSSR(reg) & SSSR_ALT_FRM_MASK;
+ val |= pxa2xx_spi_read(drv_data, SSSR) & SSSR_ALT_FRM_MASK;
- write_SSSR(val, reg);
+ pxa2xx_spi_write(drv_data, SSSR, val);
}
extern int pxa2xx_spi_flush(struct driver_data *drv_data);
static int spi_qup_io_config(struct spi_device *spi, struct spi_transfer *xfer)
{
struct spi_qup *controller = spi_master_get_devdata(spi->master);
- u32 config, iomode, mode;
+ u32 config, iomode, mode, control;
int ret, n_words, w_size;
if (spi->mode & SPI_LOOP && xfer->len > controller->in_fifo_sz) {
writel_relaxed(iomode, controller->base + QUP_IO_M_MODES);
+ control = readl_relaxed(controller->base + SPI_IO_CONTROL);
+
+ if (spi->mode & SPI_CPOL)
+ control |= SPI_IO_C_CLK_IDLE_HIGH;
+ else
+ control &= ~SPI_IO_C_CLK_IDLE_HIGH;
+
+ writel_relaxed(control, controller->base + SPI_IO_CONTROL);
+
config = readl_relaxed(controller->base + SPI_CONFIG);
if (spi->mode & SPI_LOOP)
rs->state &= ~TXBUSY;
spin_unlock_irqrestore(&rs->lock, flags);
+ rxdesc = NULL;
if (rs->rx) {
rxconf.direction = rs->dma_rx.direction;
rxconf.src_addr = rs->dma_rx.addr;
rxdesc->callback_param = rs;
}
+ txdesc = NULL;
if (rs->tx) {
txconf.direction = rs->dma_tx.direction;
txconf.dst_addr = rs->dma_tx.addr;
}
/* rx must be started before tx due to spi instinct */
- if (rs->rx) {
+ if (rxdesc) {
spin_lock_irqsave(&rs->lock, flags);
rs->state |= RXBUSY;
spin_unlock_irqrestore(&rs->lock, flags);
dma_async_issue_pending(rs->dma_rx.ch);
}
- if (rs->tx) {
+ if (txdesc) {
spin_lock_irqsave(&rs->lock, flags);
rs->state |= TXBUSY;
spin_unlock_irqrestore(&rs->lock, flags);
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
*/
#include <linux/module.h>
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
#include <linux/init.h>
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
*/
#include <linux/clk.h>
#define MDR1_SYNCMD_LR 0x30000000 /* L/R mode */
#define MDR1_SYNCAC_SHIFT 25 /* Sync Polarity (1 = Active-low) */
#define MDR1_BITLSB_SHIFT 24 /* MSB/LSB First (1 = LSB first) */
+#define MDR1_DTDL_SHIFT 20 /* Data Pin Bit Delay for MSIOF_SYNC */
+#define MDR1_SYNCDL_SHIFT 16 /* Frame Sync Signal Timing Delay */
#define MDR1_FLD_MASK 0x0000000c /* Frame Sync Signal Interval (0-3) */
#define MDR1_FLD_SHIFT 2
#define MDR1_XXSTP 0x00000001 /* Transmission/Reception Stop on FIFO */
static struct {
unsigned short div;
- unsigned short scr;
-} const sh_msiof_spi_clk_table[] = {
- { 1, SCR_BRPS( 1) | SCR_BRDV_DIV_1 },
- { 2, SCR_BRPS( 1) | SCR_BRDV_DIV_2 },
- { 4, SCR_BRPS( 1) | SCR_BRDV_DIV_4 },
- { 8, SCR_BRPS( 1) | SCR_BRDV_DIV_8 },
- { 16, SCR_BRPS( 1) | SCR_BRDV_DIV_16 },
- { 32, SCR_BRPS( 1) | SCR_BRDV_DIV_32 },
- { 64, SCR_BRPS(32) | SCR_BRDV_DIV_2 },
- { 128, SCR_BRPS(32) | SCR_BRDV_DIV_4 },
- { 256, SCR_BRPS(32) | SCR_BRDV_DIV_8 },
- { 512, SCR_BRPS(32) | SCR_BRDV_DIV_16 },
- { 1024, SCR_BRPS(32) | SCR_BRDV_DIV_32 },
+ unsigned short brdv;
+} const sh_msiof_spi_div_table[] = {
+ { 1, SCR_BRDV_DIV_1 },
+ { 2, SCR_BRDV_DIV_2 },
+ { 4, SCR_BRDV_DIV_4 },
+ { 8, SCR_BRDV_DIV_8 },
+ { 16, SCR_BRDV_DIV_16 },
+ { 32, SCR_BRDV_DIV_32 },
};
static void sh_msiof_spi_set_clk_regs(struct sh_msiof_spi_priv *p,
unsigned long parent_rate, u32 spi_hz)
{
unsigned long div = 1024;
+ u32 brps, scr;
size_t k;
if (!WARN_ON(!spi_hz || !parent_rate))
div = DIV_ROUND_UP(parent_rate, spi_hz);
- /* TODO: make more fine grained */
-
- for (k = 0; k < ARRAY_SIZE(sh_msiof_spi_clk_table); k++) {
- if (sh_msiof_spi_clk_table[k].div >= div)
+ for (k = 0; k < ARRAY_SIZE(sh_msiof_spi_div_table); k++) {
+ brps = DIV_ROUND_UP(div, sh_msiof_spi_div_table[k].div);
+ if (brps <= 32) /* max of brdv is 32 */
break;
}
- k = min_t(int, k, ARRAY_SIZE(sh_msiof_spi_clk_table) - 1);
+ k = min_t(int, k, ARRAY_SIZE(sh_msiof_spi_div_table) - 1);
- sh_msiof_write(p, TSCR, sh_msiof_spi_clk_table[k].scr);
+ scr = sh_msiof_spi_div_table[k].brdv | SCR_BRPS(brps);
+ sh_msiof_write(p, TSCR, scr);
if (!(p->chipdata->master_flags & SPI_MASTER_MUST_TX))
- sh_msiof_write(p, RSCR, sh_msiof_spi_clk_table[k].scr);
+ sh_msiof_write(p, RSCR, scr);
+}
+
+static u32 sh_msiof_get_delay_bit(u32 dtdl_or_syncdl)
+{
+ /*
+ * DTDL/SYNCDL bit : p->info->dtdl or p->info->syncdl
+ * b'000 : 0
+ * b'001 : 100
+ * b'010 : 200
+ * b'011 (SYNCDL only) : 300
+ * b'101 : 50
+ * b'110 : 150
+ */
+ if (dtdl_or_syncdl % 100)
+ return dtdl_or_syncdl / 100 + 5;
+ else
+ return dtdl_or_syncdl / 100;
+}
+
+static u32 sh_msiof_spi_get_dtdl_and_syncdl(struct sh_msiof_spi_priv *p)
+{
+ u32 val;
+
+ if (!p->info)
+ return 0;
+
+ /* check if DTDL and SYNCDL is allowed value */
+ if (p->info->dtdl > 200 || p->info->syncdl > 300) {
+ dev_warn(&p->pdev->dev, "DTDL or SYNCDL is too large\n");
+ return 0;
+ }
+
+ /* check if the sum of DTDL and SYNCDL becomes an integer value */
+ if ((p->info->dtdl + p->info->syncdl) % 100) {
+ dev_warn(&p->pdev->dev, "the sum of DTDL/SYNCDL is not good\n");
+ return 0;
+ }
+
+ val = sh_msiof_get_delay_bit(p->info->dtdl) << MDR1_DTDL_SHIFT;
+ val |= sh_msiof_get_delay_bit(p->info->syncdl) << MDR1_SYNCDL_SHIFT;
+
+ return val;
}
static void sh_msiof_spi_set_pin_regs(struct sh_msiof_spi_priv *p,
tmp = MDR1_SYNCMD_SPI | 1 << MDR1_FLD_SHIFT | MDR1_XXSTP;
tmp |= !cs_high << MDR1_SYNCAC_SHIFT;
tmp |= lsb_first << MDR1_BITLSB_SHIFT;
+ tmp |= sh_msiof_spi_get_dtdl_and_syncdl(p);
sh_msiof_write(p, TMDR1, tmp | MDR1_TRMD | TMDR1_PCON);
if (p->chipdata->master_flags & SPI_MASTER_MUST_TX) {
/* These bits are reserved if RX needs TX */
gpio_set_value(spi->cs_gpio, !(spi->mode & SPI_CS_HIGH));
- pm_runtime_put_sync(&p->pdev->dev);
+ pm_runtime_put(&p->pdev->dev);
return 0;
}
}
/* wait for tx fifo to be emptied / rx fifo to be filled */
- ret = wait_for_completion_timeout(&p->done, HZ);
- if (!ret) {
+ if (!wait_for_completion_timeout(&p->done, HZ)) {
dev_err(&p->pdev->dev, "PIO timeout\n");
ret = -ETIMEDOUT;
goto stop_reset;
}
/* wait for tx fifo to be emptied / rx fifo to be filled */
- ret = wait_for_completion_timeout(&p->done, HZ);
- if (!ret) {
+ if (!wait_for_completion_timeout(&p->done, HZ)) {
dev_err(&p->pdev->dev, "DMA timeout\n");
ret = -ETIMEDOUT;
goto stop_reset;
&info->tx_fifo_override);
of_property_read_u32(np, "renesas,rx-fifo-size",
&info->rx_fifo_override);
+ of_property_read_u32(np, "renesas,dtdl", &info->dtdl);
+ of_property_read_u32(np, "renesas,syncdl", &info->syncdl);
info->num_chipselect = num_cs;
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
*/
#include <linux/module.h>
static const struct of_device_id spi_sirfsoc_of_match[] = {
{ .compatible = "sirf,prima2-spi", },
- { .compatible = "sirf,marco-spi", },
{}
};
MODULE_DEVICE_TABLE(of, spi_sirfsoc_of_match);
--- /dev/null
+/*
+ * Copyright (c) 2008-2014 STMicroelectronics Limited
+ *
+ * Author: Angus Clark <Angus.Clark@st.com>
+ * Patrice Chotard <patrice.chotard@st.com>
+ * Lee Jones <lee.jones@linaro.org>
+ *
+ * SPI master mode controller driver, used in STMicroelectronics devices.
+ *
+ * May be copied or modified under the terms of the GNU General Public
+ * License Version 2.0 only. See linux/COPYING for more information.
+ */
+
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/pinctrl/consumer.h>
+#include <linux/platform_device.h>
+#include <linux/of.h>
+#include <linux/of_gpio.h>
+#include <linux/of_irq.h>
+#include <linux/pm_runtime.h>
+#include <linux/spi/spi.h>
+#include <linux/spi/spi_bitbang.h>
+
+/* SSC registers */
+#define SSC_BRG 0x000
+#define SSC_TBUF 0x004
+#define SSC_RBUF 0x008
+#define SSC_CTL 0x00C
+#define SSC_IEN 0x010
+#define SSC_I2C 0x018
+
+/* SSC Control */
+#define SSC_CTL_DATA_WIDTH_9 0x8
+#define SSC_CTL_DATA_WIDTH_MSK 0xf
+#define SSC_CTL_BM 0xf
+#define SSC_CTL_HB BIT(4)
+#define SSC_CTL_PH BIT(5)
+#define SSC_CTL_PO BIT(6)
+#define SSC_CTL_SR BIT(7)
+#define SSC_CTL_MS BIT(8)
+#define SSC_CTL_EN BIT(9)
+#define SSC_CTL_LPB BIT(10)
+#define SSC_CTL_EN_TX_FIFO BIT(11)
+#define SSC_CTL_EN_RX_FIFO BIT(12)
+#define SSC_CTL_EN_CLST_RX BIT(13)
+
+/* SSC Interrupt Enable */
+#define SSC_IEN_TEEN BIT(2)
+
+#define FIFO_SIZE 8
+
+struct spi_st {
+ /* SSC SPI Controller */
+ void __iomem *base;
+ struct clk *clk;
+ struct device *dev;
+
+ /* SSC SPI current transaction */
+ const u8 *tx_ptr;
+ u8 *rx_ptr;
+ u16 bytes_per_word;
+ unsigned int words_remaining;
+ unsigned int baud;
+ struct completion done;
+};
+
+static int spi_st_clk_enable(struct spi_st *spi_st)
+{
+ /*
+ * Current platforms use one of the core clocks for SPI and I2C.
+ * If we attempt to disable the clock, the system will hang.
+ *
+ * TODO: Remove this when platform supports power domains.
+ */
+ return 0;
+
+ return clk_prepare_enable(spi_st->clk);
+}
+
+static void spi_st_clk_disable(struct spi_st *spi_st)
+{
+ /*
+ * Current platforms use one of the core clocks for SPI and I2C.
+ * If we attempt to disable the clock, the system will hang.
+ *
+ * TODO: Remove this when platform supports power domains.
+ */
+ return;
+
+ clk_disable_unprepare(spi_st->clk);
+}
+
+/* Load the TX FIFO */
+static void ssc_write_tx_fifo(struct spi_st *spi_st)
+{
+ unsigned int count, i;
+ uint32_t word = 0;
+
+ if (spi_st->words_remaining > FIFO_SIZE)
+ count = FIFO_SIZE;
+ else
+ count = spi_st->words_remaining;
+
+ for (i = 0; i < count; i++) {
+ if (spi_st->tx_ptr) {
+ if (spi_st->bytes_per_word == 1) {
+ word = *spi_st->tx_ptr++;
+ } else {
+ word = *spi_st->tx_ptr++;
+ word = *spi_st->tx_ptr++ | (word << 8);
+ }
+ }
+ writel_relaxed(word, spi_st->base + SSC_TBUF);
+ }
+}
+
+/* Read the RX FIFO */
+static void ssc_read_rx_fifo(struct spi_st *spi_st)
+{
+ unsigned int count, i;
+ uint32_t word = 0;
+
+ if (spi_st->words_remaining > FIFO_SIZE)
+ count = FIFO_SIZE;
+ else
+ count = spi_st->words_remaining;
+
+ for (i = 0; i < count; i++) {
+ word = readl_relaxed(spi_st->base + SSC_RBUF);
+
+ if (spi_st->rx_ptr) {
+ if (spi_st->bytes_per_word == 1) {
+ *spi_st->rx_ptr++ = (uint8_t)word;
+ } else {
+ *spi_st->rx_ptr++ = (word >> 8);
+ *spi_st->rx_ptr++ = word & 0xff;
+ }
+ }
+ }
+ spi_st->words_remaining -= count;
+}
+
+static int spi_st_transfer_one(struct spi_master *master,
+ struct spi_device *spi, struct spi_transfer *t)
+{
+ struct spi_st *spi_st = spi_master_get_devdata(master);
+ uint32_t ctl = 0;
+
+ /* Setup transfer */
+ spi_st->tx_ptr = t->tx_buf;
+ spi_st->rx_ptr = t->rx_buf;
+
+ if (spi->bits_per_word > 8) {
+ /*
+ * Anything greater than 8 bits-per-word requires 2
+ * bytes-per-word in the RX/TX buffers
+ */
+ spi_st->bytes_per_word = 2;
+ spi_st->words_remaining = t->len / 2;
+
+ } else if (spi->bits_per_word == 8 && !(t->len & 0x1)) {
+ /*
+ * If transfer is even-length, and 8 bits-per-word, then
+ * implement as half-length 16 bits-per-word transfer
+ */
+ spi_st->bytes_per_word = 2;
+ spi_st->words_remaining = t->len / 2;
+
+ /* Set SSC_CTL to 16 bits-per-word */
+ ctl = readl_relaxed(spi_st->base + SSC_CTL);
+ writel_relaxed((ctl | 0xf), spi_st->base + SSC_CTL);
+
+ readl_relaxed(spi_st->base + SSC_RBUF);
+
+ } else {
+ spi_st->bytes_per_word = 1;
+ spi_st->words_remaining = t->len;
+ }
+
+ reinit_completion(&spi_st->done);
+
+ /* Start transfer by writing to the TX FIFO */
+ ssc_write_tx_fifo(spi_st);
+ writel_relaxed(SSC_IEN_TEEN, spi_st->base + SSC_IEN);
+
+ /* Wait for transfer to complete */
+ wait_for_completion(&spi_st->done);
+
+ /* Restore SSC_CTL if necessary */
+ if (ctl)
+ writel_relaxed(ctl, spi_st->base + SSC_CTL);
+
+ spi_finalize_current_transfer(spi->master);
+
+ return t->len;
+}
+
+static void spi_st_cleanup(struct spi_device *spi)
+{
+ int cs = spi->cs_gpio;
+
+ if (gpio_is_valid(cs))
+ devm_gpio_free(&spi->dev, cs);
+}
+
+/* the spi->mode bits understood by this driver: */
+#define MODEBITS (SPI_CPOL | SPI_CPHA | SPI_LSB_FIRST | SPI_LOOP | SPI_CS_HIGH)
+static int spi_st_setup(struct spi_device *spi)
+{
+ struct spi_st *spi_st = spi_master_get_devdata(spi->master);
+ u32 spi_st_clk, sscbrg, var;
+ u32 hz = spi->max_speed_hz;
+ int cs = spi->cs_gpio;
+ int ret;
+
+ if (!hz) {
+ dev_err(&spi->dev, "max_speed_hz unspecified\n");
+ return -EINVAL;
+ }
+
+ if (!gpio_is_valid(cs)) {
+ dev_err(&spi->dev, "%d is not a valid gpio\n", cs);
+ return -EINVAL;
+ }
+
+ if (devm_gpio_request(&spi->dev, cs, dev_name(&spi->dev))) {
+ dev_err(&spi->dev, "could not request gpio:%d\n", cs);
+ return -EINVAL;
+ }
+
+ ret = gpio_direction_output(cs, spi->mode & SPI_CS_HIGH);
+ if (ret)
+ return ret;
+
+ spi_st_clk = clk_get_rate(spi_st->clk);
+
+ /* Set SSC_BRF */
+ sscbrg = spi_st_clk / (2 * hz);
+ if (sscbrg < 0x07 || sscbrg > BIT(16)) {
+ dev_err(&spi->dev,
+ "baudrate %d outside valid range %d\n", sscbrg, hz);
+ return -EINVAL;
+ }
+
+ spi_st->baud = spi_st_clk / (2 * sscbrg);
+ if (sscbrg == BIT(16)) /* 16-bit counter wraps */
+ sscbrg = 0x0;
+
+ writel_relaxed(sscbrg, spi_st->base + SSC_BRG);
+
+ dev_dbg(&spi->dev,
+ "setting baudrate:target= %u hz, actual= %u hz, sscbrg= %u\n",
+ hz, spi_st->baud, sscbrg);
+
+ /* Set SSC_CTL and enable SSC */
+ var = readl_relaxed(spi_st->base + SSC_CTL);
+ var |= SSC_CTL_MS;
+
+ if (spi->mode & SPI_CPOL)
+ var |= SSC_CTL_PO;
+ else
+ var &= ~SSC_CTL_PO;
+
+ if (spi->mode & SPI_CPHA)
+ var |= SSC_CTL_PH;
+ else
+ var &= ~SSC_CTL_PH;
+
+ if ((spi->mode & SPI_LSB_FIRST) == 0)
+ var |= SSC_CTL_HB;
+ else
+ var &= ~SSC_CTL_HB;
+
+ if (spi->mode & SPI_LOOP)
+ var |= SSC_CTL_LPB;
+ else
+ var &= ~SSC_CTL_LPB;
+
+ var &= ~SSC_CTL_DATA_WIDTH_MSK;
+ var |= (spi->bits_per_word - 1);
+
+ var |= SSC_CTL_EN_TX_FIFO | SSC_CTL_EN_RX_FIFO;
+ var |= SSC_CTL_EN;
+
+ writel_relaxed(var, spi_st->base + SSC_CTL);
+
+ /* Clear the status register */
+ readl_relaxed(spi_st->base + SSC_RBUF);
+
+ return 0;
+}
+
+/* Interrupt fired when TX shift register becomes empty */
+static irqreturn_t spi_st_irq(int irq, void *dev_id)
+{
+ struct spi_st *spi_st = (struct spi_st *)dev_id;
+
+ /* Read RX FIFO */
+ ssc_read_rx_fifo(spi_st);
+
+ /* Fill TX FIFO */
+ if (spi_st->words_remaining) {
+ ssc_write_tx_fifo(spi_st);
+ } else {
+ /* TX/RX complete */
+ writel_relaxed(0x0, spi_st->base + SSC_IEN);
+ /*
+ * read SSC_IEN to ensure that this bit is set
+ * before re-enabling interrupt
+ */
+ readl(spi_st->base + SSC_IEN);
+ complete(&spi_st->done);
+ }
+
+ return IRQ_HANDLED;
+}
+
+static int spi_st_probe(struct platform_device *pdev)
+{
+ struct device_node *np = pdev->dev.of_node;
+ struct spi_master *master;
+ struct resource *res;
+ struct spi_st *spi_st;
+ int irq, ret = 0;
+ u32 var;
+
+ master = spi_alloc_master(&pdev->dev, sizeof(*spi_st));
+ if (!master)
+ return -ENOMEM;
+
+ master->dev.of_node = np;
+ master->mode_bits = MODEBITS;
+ master->setup = spi_st_setup;
+ master->cleanup = spi_st_cleanup;
+ master->transfer_one = spi_st_transfer_one;
+ master->bits_per_word_mask = SPI_BPW_MASK(8) | SPI_BPW_MASK(16);
+ master->auto_runtime_pm = true;
+ master->bus_num = pdev->id;
+ spi_st = spi_master_get_devdata(master);
+
+ spi_st->clk = devm_clk_get(&pdev->dev, "ssc");
+ if (IS_ERR(spi_st->clk)) {
+ dev_err(&pdev->dev, "Unable to request clock\n");
+ return PTR_ERR(spi_st->clk);
+ }
+
+ ret = spi_st_clk_enable(spi_st);
+ if (ret)
+ return ret;
+
+ init_completion(&spi_st->done);
+
+ /* Get resources */
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ spi_st->base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(spi_st->base)) {
+ ret = PTR_ERR(spi_st->base);
+ goto clk_disable;
+ }
+
+ /* Disable I2C and Reset SSC */
+ writel_relaxed(0x0, spi_st->base + SSC_I2C);
+ var = readw_relaxed(spi_st->base + SSC_CTL);
+ var |= SSC_CTL_SR;
+ writel_relaxed(var, spi_st->base + SSC_CTL);
+
+ udelay(1);
+ var = readl_relaxed(spi_st->base + SSC_CTL);
+ var &= ~SSC_CTL_SR;
+ writel_relaxed(var, spi_st->base + SSC_CTL);
+
+ /* Set SSC into slave mode before reconfiguring PIO pins */
+ var = readl_relaxed(spi_st->base + SSC_CTL);
+ var &= ~SSC_CTL_MS;
+ writel_relaxed(var, spi_st->base + SSC_CTL);
+
+ irq = irq_of_parse_and_map(np, 0);
+ if (!irq) {
+ dev_err(&pdev->dev, "IRQ missing or invalid\n");
+ ret = -EINVAL;
+ goto clk_disable;
+ }
+
+ ret = devm_request_irq(&pdev->dev, irq, spi_st_irq, 0,
+ pdev->name, spi_st);
+ if (ret) {
+ dev_err(&pdev->dev, "Failed to request irq %d\n", irq);
+ goto clk_disable;
+ }
+
+ /* by default the device is on */
+ pm_runtime_set_active(&pdev->dev);
+ pm_runtime_enable(&pdev->dev);
+
+ platform_set_drvdata(pdev, master);
+
+ ret = devm_spi_register_master(&pdev->dev, master);
+ if (ret) {
+ dev_err(&pdev->dev, "Failed to register master\n");
+ goto clk_disable;
+ }
+
+ return 0;
+
+clk_disable:
+ spi_st_clk_disable(spi_st);
+
+ return ret;
+}
+
+static int spi_st_remove(struct platform_device *pdev)
+{
+ struct spi_master *master = platform_get_drvdata(pdev);
+ struct spi_st *spi_st = spi_master_get_devdata(master);
+
+ spi_st_clk_disable(spi_st);
+
+ pinctrl_pm_select_sleep_state(&pdev->dev);
+
+ return 0;
+}
+
+#ifdef CONFIG_PM
+static int spi_st_runtime_suspend(struct device *dev)
+{
+ struct spi_master *master = dev_get_drvdata(dev);
+ struct spi_st *spi_st = spi_master_get_devdata(master);
+
+ writel_relaxed(0, spi_st->base + SSC_IEN);
+ pinctrl_pm_select_sleep_state(dev);
+
+ spi_st_clk_disable(spi_st);
+
+ return 0;
+}
+
+static int spi_st_runtime_resume(struct device *dev)
+{
+ struct spi_master *master = dev_get_drvdata(dev);
+ struct spi_st *spi_st = spi_master_get_devdata(master);
+ int ret;
+
+ ret = spi_st_clk_enable(spi_st);
+ pinctrl_pm_select_default_state(dev);
+
+ return ret;
+}
+#endif
+
+#ifdef CONFIG_PM_SLEEP
+static int spi_st_suspend(struct device *dev)
+{
+ struct spi_master *master = dev_get_drvdata(dev);
+ int ret;
+
+ ret = spi_master_suspend(master);
+ if (ret)
+ return ret;
+
+ return pm_runtime_force_suspend(dev);
+}
+
+static int spi_st_resume(struct device *dev)
+{
+ struct spi_master *master = dev_get_drvdata(dev);
+ int ret;
+
+ ret = spi_master_resume(master);
+ if (ret)
+ return ret;
+
+ return pm_runtime_force_resume(dev);
+}
+#endif
+
+static const struct dev_pm_ops spi_st_pm = {
+ SET_SYSTEM_SLEEP_PM_OPS(spi_st_suspend, spi_st_resume)
+ SET_RUNTIME_PM_OPS(spi_st_runtime_suspend, spi_st_runtime_resume, NULL)
+};
+
+static struct of_device_id stm_spi_match[] = {
+ { .compatible = "st,comms-ssc4-spi", },
+ {},
+};
+MODULE_DEVICE_TABLE(of, stm_spi_match);
+
+static struct platform_driver spi_st_driver = {
+ .driver = {
+ .name = "spi-st",
+ .pm = &spi_st_pm,
+ .of_match_table = of_match_ptr(stm_spi_match),
+ },
+ .probe = spi_st_probe,
+ .remove = spi_st_remove,
+};
+module_platform_driver(spi_st_driver);
+
+MODULE_AUTHOR("Patrice Chotard <patrice.chotard@st.com>");
+MODULE_DESCRIPTION("STM SSC SPI driver");
+MODULE_LICENSE("GPL v2");
static int qspi_write_msg(struct ti_qspi *qspi, struct spi_transfer *t)
{
- int wlen, count, ret;
+ int wlen, count;
unsigned int cmd;
const u8 *txbuf;
}
ti_qspi_write(qspi, cmd, QSPI_SPI_CMD_REG);
- ret = wait_for_completion_timeout(&qspi->transfer_complete,
- QSPI_COMPLETION_TIMEOUT);
- if (ret == 0) {
+ if (!wait_for_completion_timeout(&qspi->transfer_complete,
+ QSPI_COMPLETION_TIMEOUT)) {
dev_err(qspi->dev, "write timed out\n");
return -ETIMEDOUT;
}
static int qspi_read_msg(struct ti_qspi *qspi, struct spi_transfer *t)
{
- int wlen, count, ret;
+ int wlen, count;
unsigned int cmd;
u8 *rxbuf;
while (count) {
dev_dbg(qspi->dev, "rx cmd %08x dc %08x\n", cmd, qspi->dc);
ti_qspi_write(qspi, cmd, QSPI_SPI_CMD_REG);
- ret = wait_for_completion_timeout(&qspi->transfer_complete,
- QSPI_COMPLETION_TIMEOUT);
- if (ret == 0) {
+ if (!wait_for_completion_timeout(&qspi->transfer_complete,
+ QSPI_COMPLETION_TIMEOUT)) {
dev_err(qspi->dev, "read timed out\n");
return -ETIMEDOUT;
}
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA.
*/
#include <linux/delay.h>
#include <linux/spi/xilinx_spi.h>
#include <linux/io.h>
+#define XILINX_SPI_MAX_CS 32
+
#define XILINX_SPI_NAME "xilinx_spi"
/* Register definitions as per "OPB Serial Peripheral Interface (SPI) (v1.00e)
#define XSPI_CR_MASTER_MODE 0x04
#define XSPI_CR_CPOL 0x08
#define XSPI_CR_CPHA 0x10
-#define XSPI_CR_MODE_MASK (XSPI_CR_CPHA | XSPI_CR_CPOL)
+#define XSPI_CR_MODE_MASK (XSPI_CR_CPHA | XSPI_CR_CPOL | \
+ XSPI_CR_LSB_FIRST | XSPI_CR_LOOP)
#define XSPI_CR_TXFIFO_RESET 0x20
#define XSPI_CR_RXFIFO_RESET 0x40
#define XSPI_CR_MANUAL_SSELECT 0x80
u8 *rx_ptr; /* pointer in the Tx buffer */
const u8 *tx_ptr; /* pointer in the Rx buffer */
- int remaining_bytes; /* the number of bytes left to transfer */
- u8 bits_per_word;
+ u8 bytes_per_word;
+ int buffer_size; /* buffer size in words */
+ u32 cs_inactive; /* Level of the CS pins when inactive*/
unsigned int (*read_fn)(void __iomem *);
void (*write_fn)(u32, void __iomem *);
- void (*tx_fn)(struct xilinx_spi *);
- void (*rx_fn)(struct xilinx_spi *);
};
static void xspi_write32(u32 val, void __iomem *addr)
return ioread32be(addr);
}
-static void xspi_tx8(struct xilinx_spi *xspi)
+static void xilinx_spi_tx(struct xilinx_spi *xspi)
{
- xspi->write_fn(*xspi->tx_ptr, xspi->regs + XSPI_TXD_OFFSET);
- xspi->tx_ptr++;
-}
-
-static void xspi_tx16(struct xilinx_spi *xspi)
-{
- xspi->write_fn(*(u16 *)(xspi->tx_ptr), xspi->regs + XSPI_TXD_OFFSET);
- xspi->tx_ptr += 2;
-}
+ u32 data = 0;
-static void xspi_tx32(struct xilinx_spi *xspi)
-{
- xspi->write_fn(*(u32 *)(xspi->tx_ptr), xspi->regs + XSPI_TXD_OFFSET);
- xspi->tx_ptr += 4;
-}
-
-static void xspi_rx8(struct xilinx_spi *xspi)
-{
- u32 data = xspi->read_fn(xspi->regs + XSPI_RXD_OFFSET);
- if (xspi->rx_ptr) {
- *xspi->rx_ptr = data & 0xff;
- xspi->rx_ptr++;
+ if (!xspi->tx_ptr) {
+ xspi->write_fn(0, xspi->regs + XSPI_TXD_OFFSET);
+ return;
}
-}
-static void xspi_rx16(struct xilinx_spi *xspi)
-{
- u32 data = xspi->read_fn(xspi->regs + XSPI_RXD_OFFSET);
- if (xspi->rx_ptr) {
- *(u16 *)(xspi->rx_ptr) = data & 0xffff;
- xspi->rx_ptr += 2;
+ switch (xspi->bytes_per_word) {
+ case 1:
+ data = *(u8 *)(xspi->tx_ptr);
+ break;
+ case 2:
+ data = *(u16 *)(xspi->tx_ptr);
+ break;
+ case 4:
+ data = *(u32 *)(xspi->tx_ptr);
+ break;
}
+
+ xspi->write_fn(data, xspi->regs + XSPI_TXD_OFFSET);
+ xspi->tx_ptr += xspi->bytes_per_word;
}
-static void xspi_rx32(struct xilinx_spi *xspi)
+static void xilinx_spi_rx(struct xilinx_spi *xspi)
{
u32 data = xspi->read_fn(xspi->regs + XSPI_RXD_OFFSET);
- if (xspi->rx_ptr) {
+
+ if (!xspi->rx_ptr)
+ return;
+
+ switch (xspi->bytes_per_word) {
+ case 1:
+ *(u8 *)(xspi->rx_ptr) = data;
+ break;
+ case 2:
+ *(u16 *)(xspi->rx_ptr) = data;
+ break;
+ case 4:
*(u32 *)(xspi->rx_ptr) = data;
- xspi->rx_ptr += 4;
+ break;
}
+
+ xspi->rx_ptr += xspi->bytes_per_word;
}
static void xspi_init_hw(struct xilinx_spi *xspi)
/* Reset the SPI device */
xspi->write_fn(XIPIF_V123B_RESET_MASK,
regs_base + XIPIF_V123B_RESETR_OFFSET);
- /* Disable all the interrupts just in case */
- xspi->write_fn(0, regs_base + XIPIF_V123B_IIER_OFFSET);
- /* Enable the global IPIF interrupt */
- xspi->write_fn(XIPIF_V123B_GINTR_ENABLE,
- regs_base + XIPIF_V123B_DGIER_OFFSET);
+ /* Enable the transmit empty interrupt, which we use to determine
+ * progress on the transmission.
+ */
+ xspi->write_fn(XSPI_INTR_TX_EMPTY,
+ regs_base + XIPIF_V123B_IIER_OFFSET);
+ /* Disable the global IPIF interrupt */
+ xspi->write_fn(0, regs_base + XIPIF_V123B_DGIER_OFFSET);
/* Deselect the slave on the SPI bus */
xspi->write_fn(0xffff, regs_base + XSPI_SSR_OFFSET);
/* Disable the transmitter, enable Manual Slave Select Assertion,
* put SPI controller into master mode, and enable it */
- xspi->write_fn(XSPI_CR_TRANS_INHIBIT | XSPI_CR_MANUAL_SSELECT |
- XSPI_CR_MASTER_MODE | XSPI_CR_ENABLE | XSPI_CR_TXFIFO_RESET |
- XSPI_CR_RXFIFO_RESET, regs_base + XSPI_CR_OFFSET);
+ xspi->write_fn(XSPI_CR_MANUAL_SSELECT | XSPI_CR_MASTER_MODE |
+ XSPI_CR_ENABLE | XSPI_CR_TXFIFO_RESET | XSPI_CR_RXFIFO_RESET,
+ regs_base + XSPI_CR_OFFSET);
}
static void xilinx_spi_chipselect(struct spi_device *spi, int is_on)
{
struct xilinx_spi *xspi = spi_master_get_devdata(spi->master);
+ u16 cr;
+ u32 cs;
if (is_on == BITBANG_CS_INACTIVE) {
/* Deselect the slave on the SPI bus */
- xspi->write_fn(0xffff, xspi->regs + XSPI_SSR_OFFSET);
- } else if (is_on == BITBANG_CS_ACTIVE) {
- /* Set the SPI clock phase and polarity */
- u16 cr = xspi->read_fn(xspi->regs + XSPI_CR_OFFSET)
- & ~XSPI_CR_MODE_MASK;
- if (spi->mode & SPI_CPHA)
- cr |= XSPI_CR_CPHA;
- if (spi->mode & SPI_CPOL)
- cr |= XSPI_CR_CPOL;
- xspi->write_fn(cr, xspi->regs + XSPI_CR_OFFSET);
-
- /* We do not check spi->max_speed_hz here as the SPI clock
- * frequency is not software programmable (the IP block design
- * parameter)
- */
-
- /* Activate the chip select */
- xspi->write_fn(~(0x0001 << spi->chip_select),
- xspi->regs + XSPI_SSR_OFFSET);
+ xspi->write_fn(xspi->cs_inactive, xspi->regs + XSPI_SSR_OFFSET);
+ return;
}
+
+ /* Set the SPI clock phase and polarity */
+ cr = xspi->read_fn(xspi->regs + XSPI_CR_OFFSET) & ~XSPI_CR_MODE_MASK;
+ if (spi->mode & SPI_CPHA)
+ cr |= XSPI_CR_CPHA;
+ if (spi->mode & SPI_CPOL)
+ cr |= XSPI_CR_CPOL;
+ if (spi->mode & SPI_LSB_FIRST)
+ cr |= XSPI_CR_LSB_FIRST;
+ if (spi->mode & SPI_LOOP)
+ cr |= XSPI_CR_LOOP;
+ xspi->write_fn(cr, xspi->regs + XSPI_CR_OFFSET);
+
+ /* We do not check spi->max_speed_hz here as the SPI clock
+ * frequency is not software programmable (the IP block design
+ * parameter)
+ */
+
+ cs = xspi->cs_inactive;
+ cs ^= BIT(spi->chip_select);
+
+ /* Activate the chip select */
+ xspi->write_fn(cs, xspi->regs + XSPI_SSR_OFFSET);
}
/* spi_bitbang requires custom setup_transfer() to be defined if there is a
static int xilinx_spi_setup_transfer(struct spi_device *spi,
struct spi_transfer *t)
{
- return 0;
-}
+ struct xilinx_spi *xspi = spi_master_get_devdata(spi->master);
-static void xilinx_spi_fill_tx_fifo(struct xilinx_spi *xspi)
-{
- u8 sr;
+ if (spi->mode & SPI_CS_HIGH)
+ xspi->cs_inactive &= ~BIT(spi->chip_select);
+ else
+ xspi->cs_inactive |= BIT(spi->chip_select);
- /* Fill the Tx FIFO with as many bytes as possible */
- sr = xspi->read_fn(xspi->regs + XSPI_SR_OFFSET);
- while ((sr & XSPI_SR_TX_FULL_MASK) == 0 && xspi->remaining_bytes > 0) {
- if (xspi->tx_ptr)
- xspi->tx_fn(xspi);
- else
- xspi->write_fn(0, xspi->regs + XSPI_TXD_OFFSET);
- xspi->remaining_bytes -= xspi->bits_per_word / 8;
- sr = xspi->read_fn(xspi->regs + XSPI_SR_OFFSET);
- }
+ return 0;
}
static int xilinx_spi_txrx_bufs(struct spi_device *spi, struct spi_transfer *t)
{
struct xilinx_spi *xspi = spi_master_get_devdata(spi->master);
- u32 ipif_ier;
+ int remaining_words; /* the number of words left to transfer */
+ bool use_irq = false;
+ u16 cr = 0;
/* We get here with transmitter inhibited */
xspi->tx_ptr = t->tx_buf;
xspi->rx_ptr = t->rx_buf;
- xspi->remaining_bytes = t->len;
+ remaining_words = t->len / xspi->bytes_per_word;
reinit_completion(&xspi->done);
+ if (xspi->irq >= 0 && remaining_words > xspi->buffer_size) {
+ use_irq = true;
+ xspi->write_fn(XSPI_INTR_TX_EMPTY,
+ xspi->regs + XIPIF_V123B_IISR_OFFSET);
+ /* Enable the global IPIF interrupt */
+ xspi->write_fn(XIPIF_V123B_GINTR_ENABLE,
+ xspi->regs + XIPIF_V123B_DGIER_OFFSET);
+ /* Inhibit irq to avoid spurious irqs on tx_empty*/
+ cr = xspi->read_fn(xspi->regs + XSPI_CR_OFFSET);
+ xspi->write_fn(cr | XSPI_CR_TRANS_INHIBIT,
+ xspi->regs + XSPI_CR_OFFSET);
+ }
- /* Enable the transmit empty interrupt, which we use to determine
- * progress on the transmission.
- */
- ipif_ier = xspi->read_fn(xspi->regs + XIPIF_V123B_IIER_OFFSET);
- xspi->write_fn(ipif_ier | XSPI_INTR_TX_EMPTY,
- xspi->regs + XIPIF_V123B_IIER_OFFSET);
+ while (remaining_words) {
+ int n_words, tx_words, rx_words;
- for (;;) {
- u16 cr;
- u8 sr;
+ n_words = min(remaining_words, xspi->buffer_size);
- xilinx_spi_fill_tx_fifo(xspi);
+ tx_words = n_words;
+ while (tx_words--)
+ xilinx_spi_tx(xspi);
/* Start the transfer by not inhibiting the transmitter any
* longer
*/
- cr = xspi->read_fn(xspi->regs + XSPI_CR_OFFSET) &
- ~XSPI_CR_TRANS_INHIBIT;
- xspi->write_fn(cr, xspi->regs + XSPI_CR_OFFSET);
- wait_for_completion(&xspi->done);
+ if (use_irq) {
+ xspi->write_fn(cr, xspi->regs + XSPI_CR_OFFSET);
+ wait_for_completion(&xspi->done);
+ } else
+ while (!(xspi->read_fn(xspi->regs + XSPI_SR_OFFSET) &
+ XSPI_SR_TX_EMPTY_MASK))
+ ;
/* A transmit has just completed. Process received data and
* check for more data to transmit. Always inhibit the
* transmitter while the Isr refills the transmit register/FIFO,
* or make sure it is stopped if we're done.
*/
- cr = xspi->read_fn(xspi->regs + XSPI_CR_OFFSET);
- xspi->write_fn(cr | XSPI_CR_TRANS_INHIBIT,
+ if (use_irq)
+ xspi->write_fn(cr | XSPI_CR_TRANS_INHIBIT,
xspi->regs + XSPI_CR_OFFSET);
/* Read out all the data from the Rx FIFO */
- sr = xspi->read_fn(xspi->regs + XSPI_SR_OFFSET);
- while ((sr & XSPI_SR_RX_EMPTY_MASK) == 0) {
- xspi->rx_fn(xspi);
- sr = xspi->read_fn(xspi->regs + XSPI_SR_OFFSET);
- }
-
- /* See if there is more data to send */
- if (xspi->remaining_bytes <= 0)
- break;
+ rx_words = n_words;
+ while (rx_words--)
+ xilinx_spi_rx(xspi);
+
+ remaining_words -= n_words;
}
- /* Disable the transmit empty interrupt */
- xspi->write_fn(ipif_ier, xspi->regs + XIPIF_V123B_IIER_OFFSET);
+ if (use_irq)
+ xspi->write_fn(0, xspi->regs + XIPIF_V123B_DGIER_OFFSET);
- return t->len - xspi->remaining_bytes;
+ return t->len;
}
return IRQ_HANDLED;
}
+static int xilinx_spi_find_buffer_size(struct xilinx_spi *xspi)
+{
+ u8 sr;
+ int n_words = 0;
+
+ /*
+ * Before the buffer_size detection we reset the core
+ * to make sure we start with a clean state.
+ */
+ xspi->write_fn(XIPIF_V123B_RESET_MASK,
+ xspi->regs + XIPIF_V123B_RESETR_OFFSET);
+
+ /* Fill the Tx FIFO with as many words as possible */
+ do {
+ xspi->write_fn(0, xspi->regs + XSPI_TXD_OFFSET);
+ sr = xspi->read_fn(xspi->regs + XSPI_SR_OFFSET);
+ n_words++;
+ } while (!(sr & XSPI_SR_TX_FULL_MASK));
+
+ return n_words;
+}
+
static const struct of_device_id xilinx_spi_of_match[] = {
{ .compatible = "xlnx,xps-spi-2.00.a", },
{ .compatible = "xlnx,xps-spi-2.00.b", },
return -EINVAL;
}
+ if (num_cs > XILINX_SPI_MAX_CS) {
+ dev_err(&pdev->dev, "Invalid number of spi slaves\n");
+ return -EINVAL;
+ }
+
master = spi_alloc_master(&pdev->dev, sizeof(struct xilinx_spi));
if (!master)
return -ENODEV;
/* the spi->mode bits understood by this driver: */
- master->mode_bits = SPI_CPOL | SPI_CPHA;
+ master->mode_bits = SPI_CPOL | SPI_CPHA | SPI_LSB_FIRST | SPI_LOOP |
+ SPI_CS_HIGH;
xspi = spi_master_get_devdata(master);
+ xspi->cs_inactive = 0xffffffff;
xspi->bitbang.master = master;
xspi->bitbang.chipselect = xilinx_spi_chipselect;
xspi->bitbang.setup_transfer = xilinx_spi_setup_transfer;
}
master->bits_per_word_mask = SPI_BPW_MASK(bits_per_word);
- xspi->bits_per_word = bits_per_word;
- if (xspi->bits_per_word == 8) {
- xspi->tx_fn = xspi_tx8;
- xspi->rx_fn = xspi_rx8;
- } else if (xspi->bits_per_word == 16) {
- xspi->tx_fn = xspi_tx16;
- xspi->rx_fn = xspi_rx16;
- } else if (xspi->bits_per_word == 32) {
- xspi->tx_fn = xspi_tx32;
- xspi->rx_fn = xspi_rx32;
- } else {
- ret = -EINVAL;
- goto put_master;
- }
-
- /* SPI controller initializations */
- xspi_init_hw(xspi);
+ xspi->bytes_per_word = bits_per_word / 8;
+ xspi->buffer_size = xilinx_spi_find_buffer_size(xspi);
xspi->irq = platform_get_irq(pdev, 0);
- if (xspi->irq < 0) {
- ret = xspi->irq;
- goto put_master;
+ if (xspi->irq >= 0) {
+ /* Register for SPI Interrupt */
+ ret = devm_request_irq(&pdev->dev, xspi->irq, xilinx_spi_irq, 0,
+ dev_name(&pdev->dev), xspi);
+ if (ret)
+ goto put_master;
}
- /* Register for SPI Interrupt */
- ret = devm_request_irq(&pdev->dev, xspi->irq, xilinx_spi_irq, 0,
- dev_name(&pdev->dev), xspi);
- if (ret)
- goto put_master;
+ /* SPI controller initializations */
+ xspi_init_hw(xspi);
ret = spi_bitbang_start(&xspi->bitbang);
if (ret) {
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
#include <linux/kernel.h>
struct spi_transfer *xfer;
bool keep_cs = false;
int ret = 0;
- int ms = 1;
+ unsigned long ms = 1;
spi_set_cs(msg->spi, true);
EXPORT_SYMBOL_GPL(spi_finalize_current_transfer);
/**
- * spi_pump_messages - kthread work function which processes spi message queue
- * @work: pointer to kthread work struct contained in the master struct
+ * __spi_pump_messages - function which processes spi message queue
+ * @master: master to process queue for
+ * @in_kthread: true if we are in the context of the message pump thread
*
* This function checks if there is any spi message in the queue that
* needs processing and if so call out to the driver to initialize hardware
* and transfer each message.
*
+ * Note that it is called both from the kthread itself and also from
+ * inside spi_sync(); the queue extraction handling at the top of the
+ * function should deal with this safely.
*/
-static void spi_pump_messages(struct kthread_work *work)
+static void __spi_pump_messages(struct spi_master *master, bool in_kthread)
{
- struct spi_master *master =
- container_of(work, struct spi_master, pump_messages);
unsigned long flags;
bool was_busy = false;
int ret;
- /* Lock queue and check for queue work */
+ /* Lock queue */
spin_lock_irqsave(&master->queue_lock, flags);
+
+ /* Make sure we are not already running a message */
+ if (master->cur_msg) {
+ spin_unlock_irqrestore(&master->queue_lock, flags);
+ return;
+ }
+
+ /* If another context is idling the device then defer */
+ if (master->idling) {
+ queue_kthread_work(&master->kworker, &master->pump_messages);
+ spin_unlock_irqrestore(&master->queue_lock, flags);
+ return;
+ }
+
+ /* Check if the queue is idle */
if (list_empty(&master->queue) || !master->running) {
if (!master->busy) {
spin_unlock_irqrestore(&master->queue_lock, flags);
return;
}
+
+ /* Only do teardown in the thread */
+ if (!in_kthread) {
+ queue_kthread_work(&master->kworker,
+ &master->pump_messages);
+ spin_unlock_irqrestore(&master->queue_lock, flags);
+ return;
+ }
+
master->busy = false;
+ master->idling = true;
spin_unlock_irqrestore(&master->queue_lock, flags);
+
kfree(master->dummy_rx);
master->dummy_rx = NULL;
kfree(master->dummy_tx);
pm_runtime_put_autosuspend(master->dev.parent);
}
trace_spi_master_idle(master);
- return;
- }
- /* Make sure we are not already running a message */
- if (master->cur_msg) {
+ spin_lock_irqsave(&master->queue_lock, flags);
+ master->idling = false;
spin_unlock_irqrestore(&master->queue_lock, flags);
return;
}
+
/* Extract head of queue */
master->cur_msg =
list_first_entry(&master->queue, struct spi_message, queue);
}
}
+/**
+ * spi_pump_messages - kthread work function which processes spi message queue
+ * @work: pointer to kthread work struct contained in the master struct
+ */
+static void spi_pump_messages(struct kthread_work *work)
+{
+ struct spi_master *master =
+ container_of(work, struct spi_master, pump_messages);
+
+ __spi_pump_messages(master, true);
+}
+
static int spi_init_queue(struct spi_master *master)
{
struct sched_param param = { .sched_priority = MAX_RT_PRIO - 1 };
- INIT_LIST_HEAD(&master->queue);
- spin_lock_init(&master->queue_lock);
-
master->running = false;
master->busy = false;
return 0;
}
-/**
- * spi_queued_transfer - transfer function for queued transfers
- * @spi: spi device which is requesting transfer
- * @msg: spi message which is to handled is queued to driver queue
- */
-static int spi_queued_transfer(struct spi_device *spi, struct spi_message *msg)
+static int __spi_queued_transfer(struct spi_device *spi,
+ struct spi_message *msg,
+ bool need_pump)
{
struct spi_master *master = spi->master;
unsigned long flags;
msg->status = -EINPROGRESS;
list_add_tail(&msg->queue, &master->queue);
- if (!master->busy)
+ if (!master->busy && need_pump)
queue_kthread_work(&master->kworker, &master->pump_messages);
spin_unlock_irqrestore(&master->queue_lock, flags);
return 0;
}
+/**
+ * spi_queued_transfer - transfer function for queued transfers
+ * @spi: spi device which is requesting transfer
+ * @msg: spi message which is to handled is queued to driver queue
+ */
+static int spi_queued_transfer(struct spi_device *spi, struct spi_message *msg)
+{
+ return __spi_queued_transfer(spi, msg, true);
+}
+
static int spi_master_initialize_queue(struct spi_master *master)
{
int ret;
dynamic = 1;
}
+ INIT_LIST_HEAD(&master->queue);
+ spin_lock_init(&master->queue_lock);
spin_lock_init(&master->bus_lock_spinlock);
mutex_init(&master->bus_lock_mutex);
master->bus_lock_flag = 0;
DECLARE_COMPLETION_ONSTACK(done);
int status;
struct spi_master *master = spi->master;
+ unsigned long flags;
+
+ status = __spi_validate(spi, message);
+ if (status != 0)
+ return status;
message->complete = spi_complete;
message->context = &done;
+ message->spi = spi;
if (!bus_locked)
mutex_lock(&master->bus_lock_mutex);
- status = spi_async_locked(spi, message);
+ /* If we're not using the legacy transfer method then we will
+ * try to transfer in the calling context so special case.
+ * This code would be less tricky if we could remove the
+ * support for driver implemented message queues.
+ */
+ if (master->transfer == spi_queued_transfer) {
+ spin_lock_irqsave(&master->bus_lock_spinlock, flags);
+
+ trace_spi_message_submit(message);
+
+ status = __spi_queued_transfer(spi, message, false);
+
+ spin_unlock_irqrestore(&master->bus_lock_spinlock, flags);
+ } else {
+ status = spi_async_locked(spi, message);
+ }
if (!bus_locked)
mutex_unlock(&master->bus_lock_mutex);
if (status == 0) {
+ /* Push out the messages in the calling context if we
+ * can.
+ */
+ if (master->transfer == spi_queued_transfer)
+ __spi_pump_messages(master, false);
+
wait_for_completion(&done);
status = message->status;
}
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
#include <linux/init.h>
return status;
}
+static struct spi_ioc_transfer *
+spidev_get_ioc_message(unsigned int cmd, struct spi_ioc_transfer __user *u_ioc,
+ unsigned *n_ioc)
+{
+ struct spi_ioc_transfer *ioc;
+ u32 tmp;
+
+ /* Check type, command number and direction */
+ if (_IOC_TYPE(cmd) != SPI_IOC_MAGIC
+ || _IOC_NR(cmd) != _IOC_NR(SPI_IOC_MESSAGE(0))
+ || _IOC_DIR(cmd) != _IOC_WRITE)
+ return ERR_PTR(-ENOTTY);
+
+ tmp = _IOC_SIZE(cmd);
+ if ((tmp % sizeof(struct spi_ioc_transfer)) != 0)
+ return ERR_PTR(-EINVAL);
+ *n_ioc = tmp / sizeof(struct spi_ioc_transfer);
+ if (*n_ioc == 0)
+ return NULL;
+
+ /* copy into scratch area */
+ ioc = kmalloc(tmp, GFP_KERNEL);
+ if (!ioc)
+ return ERR_PTR(-ENOMEM);
+ if (__copy_from_user(ioc, u_ioc, tmp)) {
+ kfree(ioc);
+ return ERR_PTR(-EFAULT);
+ }
+ return ioc;
+}
+
static long
spidev_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
{
default:
/* segmented and/or full-duplex I/O request */
- if (_IOC_NR(cmd) != _IOC_NR(SPI_IOC_MESSAGE(0))
- || _IOC_DIR(cmd) != _IOC_WRITE) {
- retval = -ENOTTY;
- break;
- }
-
- tmp = _IOC_SIZE(cmd);
- if ((tmp % sizeof(struct spi_ioc_transfer)) != 0) {
- retval = -EINVAL;
- break;
- }
- n_ioc = tmp / sizeof(struct spi_ioc_transfer);
- if (n_ioc == 0)
- break;
-
- /* copy into scratch area */
- ioc = kmalloc(tmp, GFP_KERNEL);
- if (!ioc) {
- retval = -ENOMEM;
- break;
- }
- if (__copy_from_user(ioc, (void __user *)arg, tmp)) {
- kfree(ioc);
- retval = -EFAULT;
+ /* Check message and copy into scratch area */
+ ioc = spidev_get_ioc_message(cmd,
+ (struct spi_ioc_transfer __user *)arg, &n_ioc);
+ if (IS_ERR(ioc)) {
+ retval = PTR_ERR(ioc);
break;
}
+ if (!ioc)
+ break; /* n_ioc is also 0 */
/* translate to spi_message, execute */
retval = spidev_message(spidev, ioc, n_ioc);
}
#ifdef CONFIG_COMPAT
+static long
+spidev_compat_ioc_message(struct file *filp, unsigned int cmd,
+ unsigned long arg)
+{
+ struct spi_ioc_transfer __user *u_ioc;
+ int retval = 0;
+ struct spidev_data *spidev;
+ struct spi_device *spi;
+ unsigned n_ioc, n;
+ struct spi_ioc_transfer *ioc;
+
+ u_ioc = (struct spi_ioc_transfer __user *) compat_ptr(arg);
+ if (!access_ok(VERIFY_READ, u_ioc, _IOC_SIZE(cmd)))
+ return -EFAULT;
+
+ /* guard against device removal before, or while,
+ * we issue this ioctl.
+ */
+ spidev = filp->private_data;
+ spin_lock_irq(&spidev->spi_lock);
+ spi = spi_dev_get(spidev->spi);
+ spin_unlock_irq(&spidev->spi_lock);
+
+ if (spi == NULL)
+ return -ESHUTDOWN;
+
+ /* SPI_IOC_MESSAGE needs the buffer locked "normally" */
+ mutex_lock(&spidev->buf_lock);
+
+ /* Check message and copy into scratch area */
+ ioc = spidev_get_ioc_message(cmd, u_ioc, &n_ioc);
+ if (IS_ERR(ioc)) {
+ retval = PTR_ERR(ioc);
+ goto done;
+ }
+ if (!ioc)
+ goto done; /* n_ioc is also 0 */
+
+ /* Convert buffer pointers */
+ for (n = 0; n < n_ioc; n++) {
+ ioc[n].rx_buf = (uintptr_t) compat_ptr(ioc[n].rx_buf);
+ ioc[n].tx_buf = (uintptr_t) compat_ptr(ioc[n].tx_buf);
+ }
+
+ /* translate to spi_message, execute */
+ retval = spidev_message(spidev, ioc, n_ioc);
+ kfree(ioc);
+
+done:
+ mutex_unlock(&spidev->buf_lock);
+ spi_dev_put(spi);
+ return retval;
+}
+
static long
spidev_compat_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
{
+ if (_IOC_TYPE(cmd) == SPI_IOC_MAGIC
+ && _IOC_NR(cmd) == _IOC_NR(SPI_IOC_MESSAGE(0))
+ && _IOC_DIR(cmd) == _IOC_WRITE)
+ return spidev_compat_ioc_message(filp, cmd, arg);
+
return spidev_ioctl(filp, cmd, (unsigned long)compat_ptr(arg));
}
#else
return 0;
}
- if (cfio->fault.ft_flags & VM_FAULT_SIGBUS) {
+ if (cfio->fault.ft_flags & (VM_FAULT_SIGBUS | VM_FAULT_SIGSEGV)) {
CDEBUG(D_PAGE, "got addr %p - SIGBUS\n", vmf->virtual_address);
return -EFAULT;
}
static const struct mfd_cell nvec_devices[] = {
{
.name = "nvec-kbd",
- .id = 1,
},
{
.name = "nvec-mouse",
- .id = 1,
},
{
.name = "nvec-power",
- .id = 1,
+ .id = 0,
},
{
.name = "nvec-power",
- .id = 2,
+ .id = 1,
},
{
.name = "nvec-paz00",
- .id = 1,
},
};
nvec_msg_free(nvec, msg);
}
- ret = mfd_add_devices(nvec->dev, -1, nvec_devices,
+ ret = mfd_add_devices(nvec->dev, 0, nvec_devices,
ARRAY_SIZE(nvec_devices), NULL, 0, NULL);
if (ret)
dev_err(nvec->dev, "error adding subdevices\n");
le16_to_cpu(dev->descriptor.idProduct) == 0xbadd))
return 0;
+ /* OTG PET device is always targeted (see OTG 2.0 ECN 6.4.2) */
+ if ((le16_to_cpu(dev->descriptor.idVendor) == 0x1a0a &&
+ le16_to_cpu(dev->descriptor.idProduct) == 0x0200))
+ return 1;
+
/* NOTE: can't use usb_match_id() since interface caches
* aren't set up yet. this is cut/paste from that code.
*/
{ USB_DEVICE(0x0b05, 0x17e0), .driver_info =
USB_QUIRK_IGNORE_REMOTE_WAKEUP },
+ /* Protocol and OTG Electrical Test Device */
+ { USB_DEVICE(0x1a0a, 0x0200), .driver_info =
+ USB_QUIRK_LINEAR_UFRAME_INTR_BINTERVAL },
+
{ } /* terminating entry must be last */
};
u32 gintsts;
irqreturn_t retval = IRQ_NONE;
+ spin_lock(&hsotg->lock);
+
if (!dwc2_is_controller_alive(hsotg)) {
dev_warn(hsotg->dev, "Controller is dead\n");
goto out;
}
- spin_lock(&hsotg->lock);
-
gintsts = dwc2_read_common_intr(hsotg);
if (gintsts & ~GINTSTS_PRTINT)
retval = IRQ_HANDLED;
}
}
- spin_unlock(&hsotg->lock);
out:
+ spin_unlock(&hsotg->lock);
return retval;
}
EXPORT_SYMBOL_GPL(dwc2_handle_common_intr);
return phy;
}
- return ERR_PTR(-EPROBE_DEFER);
+ return ERR_PTR(-ENODEV);
}
static struct usb_phy *__usb_find_phy_dev(struct device *dev,
UNUSUAL_DEV( 0x04e6, 0x000f, 0x0000, 0x9999,
"SCM Microsystems",
"eUSB SCSI Adapter (Bus Powered)",
- USB_SC_DEVICE, USB_PR_DEVICE, usb_stor_euscsi_init,
+ USB_SC_SCSI, USB_PR_BULK, usb_stor_euscsi_init,
US_FL_SCM_MULT_TARG ),
UNUSUAL_DEV( 0x04e6, 0x0101, 0x0200, 0x0200,
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_IGNORE_RESIDUE | US_FL_SANE_SENSE ),
+/* Reported by Dmitry Nezhevenko <dion@dion.org.ua> */
+UNUSUAL_DEV( 0x152d, 0x2566, 0x0114, 0x0114,
+ "JMicron",
+ "USB to ATA/ATAPI Bridge",
+ USB_SC_DEVICE, USB_PR_DEVICE, NULL,
+ US_FL_BROKEN_FUA ),
+
/* Entrega Technologies U1-SC25 (later Xircom PortGear PGSCSI)
* and Mac USB Dock USB-SCSI */
UNUSUAL_DEV( 0x1645, 0x0007, 0x0100, 0x0133,
"External HDD",
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_IGNORE_UAS),
+
+/* Reported-by: Richard Henderson <rth@redhat.com> */
+UNUSUAL_DEV(0x4971, 0x8017, 0x0000, 0x9999,
+ "SimpleTech",
+ "External HDD",
+ USB_SC_DEVICE, USB_PR_DEVICE, NULL,
+ US_FL_NO_REPORT_OPCODES),
break;
}
/* TODO: Should check and handle checksum. */
+
+ hdr.num_buffers = cpu_to_vhost16(vq, headcount);
if (likely(mergeable) &&
- memcpy_toiovecend(nvq->hdr, (unsigned char *)&headcount,
+ memcpy_toiovecend(nvq->hdr, (void *)&hdr.num_buffers,
offsetof(typeof(hdr), num_buffers),
sizeof hdr.num_buffers)) {
vq_err(vq, "Failed num_buffers write");
long ret = 0;
int copy_ret;
+ /*
+ * The mutex can block and wake us up and that will cause
+ * wait_event_interruptible_hrtimeout() to schedule without sleeping
+ * and repeat. This should be rare enough that it doesn't cause
+ * peformance issues. See the comment in read_events() for more detail.
+ */
+ sched_annotate_sleep();
mutex_lock(&ctx->ring_lock);
/* Access to ->ring_pages here is protected by ctx->ring_lock. */
select LZO_DECOMPRESS
select RAID6_PQ
select XOR_BLOCKS
+ select SRCU
help
Btrfs is a general purpose copy-on-write filesystem with extents,
path->search_commit_root = 1;
path->skip_locking = 1;
+ ppath->search_commit_root = 1;
+ ppath->skip_locking = 1;
/*
* trigger the readahead for extent tree csum tree and wait for
* completion. During readahead, the scrub is officially paused
}
if (log_root_tree->log_transid_committed >= root_log_ctx.log_transid) {
+ blk_finish_plug(&plug);
mutex_unlock(&log_root_tree->log_mutex);
ret = root_log_ctx.log_ret;
goto out;
*flags = CIFSSEC_MUST_NTLMV2;
else if ((*flags & CIFSSEC_MUST_NTLM) == CIFSSEC_MUST_NTLM)
*flags = CIFSSEC_MUST_NTLM;
- else if ((*flags & CIFSSEC_MUST_LANMAN) == CIFSSEC_MUST_LANMAN)
+ else if (CIFSSEC_MUST_LANMAN &&
+ (*flags & CIFSSEC_MUST_LANMAN) == CIFSSEC_MUST_LANMAN)
*flags = CIFSSEC_MUST_LANMAN;
- else if ((*flags & CIFSSEC_MUST_PLNTXT) == CIFSSEC_MUST_PLNTXT)
+ else if (CIFSSEC_MUST_PLNTXT &&
+ (*flags & CIFSSEC_MUST_PLNTXT) == CIFSSEC_MUST_PLNTXT)
*flags = CIFSSEC_MUST_PLNTXT;
*flags |= signflags;
struct cifsLockInfo *li, *tmp;
struct cifs_fid fid;
struct cifs_pending_open open;
+ bool oplock_break_cancelled;
spin_lock(&cifs_file_list_lock);
if (--cifs_file->count > 0) {
}
spin_unlock(&cifs_file_list_lock);
- cancel_work_sync(&cifs_file->oplock_break);
+ oplock_break_cancelled = cancel_work_sync(&cifs_file->oplock_break);
if (!tcon->need_reconnect && !cifs_file->invalidHandle) {
struct TCP_Server_Info *server = tcon->ses->server;
_free_xid(xid);
}
+ if (oplock_break_cancelled)
+ cifs_done_oplock_break(cifsi);
+
cifs_del_pending_open(&open);
/*
}
rc = mdfour(p16, (unsigned char *) wpwd, len * sizeof(__le16));
- memset(wpwd, 0, 129 * sizeof(__le16));
+ memzero_explicit(wpwd, sizeof(wpwd));
return rc;
}
static int gfs2_adjust_quota(struct gfs2_inode *ip, loff_t loc,
s64 change, struct gfs2_quota_data *qd,
- struct fs_disk_quota *fdq)
+ struct qc_dqblk *fdq)
{
struct inode *inode = &ip->i_inode;
struct gfs2_sbd *sdp = GFS2_SB(inode);
be64_add_cpu(&q.qu_value, change);
qd->qd_qb.qb_value = q.qu_value;
if (fdq) {
- if (fdq->d_fieldmask & FS_DQ_BSOFT) {
- q.qu_warn = cpu_to_be64(fdq->d_blk_softlimit >> sdp->sd_fsb2bb_shift);
+ if (fdq->d_fieldmask & QC_SPC_SOFT) {
+ q.qu_warn = cpu_to_be64(fdq->d_spc_softlimit >> sdp->sd_sb.sb_bsize_shift);
qd->qd_qb.qb_warn = q.qu_warn;
}
- if (fdq->d_fieldmask & FS_DQ_BHARD) {
- q.qu_limit = cpu_to_be64(fdq->d_blk_hardlimit >> sdp->sd_fsb2bb_shift);
+ if (fdq->d_fieldmask & QC_SPC_HARD) {
+ q.qu_limit = cpu_to_be64(fdq->d_spc_hardlimit >> sdp->sd_sb.sb_bsize_shift);
qd->qd_qb.qb_limit = q.qu_limit;
}
- if (fdq->d_fieldmask & FS_DQ_BCOUNT) {
- q.qu_value = cpu_to_be64(fdq->d_bcount >> sdp->sd_fsb2bb_shift);
+ if (fdq->d_fieldmask & QC_SPACE) {
+ q.qu_value = cpu_to_be64(fdq->d_space >> sdp->sd_sb.sb_bsize_shift);
qd->qd_qb.qb_value = q.qu_value;
}
}
}
static int gfs2_get_dqblk(struct super_block *sb, struct kqid qid,
- struct fs_disk_quota *fdq)
+ struct qc_dqblk *fdq)
{
struct gfs2_sbd *sdp = sb->s_fs_info;
struct gfs2_quota_lvb *qlvb;
struct gfs2_holder q_gh;
int error;
- memset(fdq, 0, sizeof(struct fs_disk_quota));
+ memset(fdq, 0, sizeof(*fdq));
if (sdp->sd_args.ar_quota == GFS2_QUOTA_OFF)
return -ESRCH; /* Crazy XFS error code */
goto out;
qlvb = (struct gfs2_quota_lvb *)qd->qd_gl->gl_lksb.sb_lvbptr;
- fdq->d_version = FS_DQUOT_VERSION;
- fdq->d_flags = (qid.type == USRQUOTA) ? FS_USER_QUOTA : FS_GROUP_QUOTA;
- fdq->d_id = from_kqid_munged(current_user_ns(), qid);
- fdq->d_blk_hardlimit = be64_to_cpu(qlvb->qb_limit) << sdp->sd_fsb2bb_shift;
- fdq->d_blk_softlimit = be64_to_cpu(qlvb->qb_warn) << sdp->sd_fsb2bb_shift;
- fdq->d_bcount = be64_to_cpu(qlvb->qb_value) << sdp->sd_fsb2bb_shift;
+ fdq->d_spc_hardlimit = be64_to_cpu(qlvb->qb_limit) << sdp->sd_sb.sb_bsize_shift;
+ fdq->d_spc_softlimit = be64_to_cpu(qlvb->qb_warn) << sdp->sd_sb.sb_bsize_shift;
+ fdq->d_space = be64_to_cpu(qlvb->qb_value) << sdp->sd_sb.sb_bsize_shift;
gfs2_glock_dq_uninit(&q_gh);
out:
}
/* GFS2 only supports a subset of the XFS fields */
-#define GFS2_FIELDMASK (FS_DQ_BSOFT|FS_DQ_BHARD|FS_DQ_BCOUNT)
+#define GFS2_FIELDMASK (QC_SPC_SOFT|QC_SPC_HARD|QC_SPACE)
static int gfs2_set_dqblk(struct super_block *sb, struct kqid qid,
- struct fs_disk_quota *fdq)
+ struct qc_dqblk *fdq)
{
struct gfs2_sbd *sdp = sb->s_fs_info;
struct gfs2_inode *ip = GFS2_I(sdp->sd_quota_inode);
goto out_i;
/* If nothing has changed, this is a no-op */
- if ((fdq->d_fieldmask & FS_DQ_BSOFT) &&
- ((fdq->d_blk_softlimit >> sdp->sd_fsb2bb_shift) == be64_to_cpu(qd->qd_qb.qb_warn)))
- fdq->d_fieldmask ^= FS_DQ_BSOFT;
+ if ((fdq->d_fieldmask & QC_SPC_SOFT) &&
+ ((fdq->d_spc_softlimit >> sdp->sd_sb.sb_bsize_shift) == be64_to_cpu(qd->qd_qb.qb_warn)))
+ fdq->d_fieldmask ^= QC_SPC_SOFT;
- if ((fdq->d_fieldmask & FS_DQ_BHARD) &&
- ((fdq->d_blk_hardlimit >> sdp->sd_fsb2bb_shift) == be64_to_cpu(qd->qd_qb.qb_limit)))
- fdq->d_fieldmask ^= FS_DQ_BHARD;
+ if ((fdq->d_fieldmask & QC_SPC_HARD) &&
+ ((fdq->d_spc_hardlimit >> sdp->sd_sb.sb_bsize_shift) == be64_to_cpu(qd->qd_qb.qb_limit)))
+ fdq->d_fieldmask ^= QC_SPC_HARD;
- if ((fdq->d_fieldmask & FS_DQ_BCOUNT) &&
- ((fdq->d_bcount >> sdp->sd_fsb2bb_shift) == be64_to_cpu(qd->qd_qb.qb_value)))
- fdq->d_fieldmask ^= FS_DQ_BCOUNT;
+ if ((fdq->d_fieldmask & QC_SPACE) &&
+ ((fdq->d_space >> sdp->sd_sb.sb_bsize_shift) == be64_to_cpu(qd->qd_qb.qb_value)))
+ fdq->d_fieldmask ^= QC_SPACE;
if (fdq->d_fieldmask == 0)
goto out_i;
*/
ssize_t nfs_direct_IO(int rw, struct kiocb *iocb, struct iov_iter *iter, loff_t pos)
{
+ struct inode *inode = iocb->ki_filp->f_mapping->host;
+
+ /* we only support swap file calling nfs_direct_IO */
+ if (!IS_SWAPFILE(inode))
+ return 0;
+
#ifndef CONFIG_NFS_SWAP
dprintk("NFS: nfs_direct_IO (%pD) off/no(%Ld/%lu) EINVAL\n",
iocb->ki_filp, (long long) pos, iter->nr_segs);
nfs_attr_check_mountpoint(sb, fattr);
- if (((fattr->valid & NFS_ATTR_FATTR_FILEID) == 0) &&
- !nfs_attr_use_mounted_on_fileid(fattr))
+ if (nfs_attr_use_mounted_on_fileid(fattr))
+ fattr->fileid = fattr->mounted_on_fileid;
+ else if ((fattr->valid & NFS_ATTR_FATTR_FILEID) == 0)
goto out_no_inode;
if ((fattr->valid & NFS_ATTR_FATTR_TYPE) == 0)
goto out_no_inode;
(((fattr->valid & NFS_ATTR_FATTR_MOUNTPOINT) == 0) &&
((fattr->valid & NFS_ATTR_FATTR_V4_REFERRAL) == 0)))
return 0;
-
- fattr->fileid = fattr->mounted_on_fileid;
return 1;
}
prev = pos;
status = nfs_wait_client_init_complete(pos);
- if (status == 0) {
+ if (pos->cl_cons_state == NFS_CS_SESSION_INITING) {
nfs4_schedule_lease_recovery(pos);
status = nfs4_wait_clnt_recover(pos);
}
* @ti_save: Backup of journal_info field of task_struct
* @ti_flags: Flags
* @ti_count: Nest level
- * @ti_garbage: List of inode to be put when releasing semaphore
*/
struct nilfs_transaction_info {
u32 ti_magic;
one of other filesystems has a bug. */
unsigned short ti_flags;
unsigned short ti_count;
- struct list_head ti_garbage;
};
/* ti_magic */
ti->ti_count = 0;
ti->ti_save = cur_ti;
ti->ti_magic = NILFS_TI_MAGIC;
- INIT_LIST_HEAD(&ti->ti_garbage);
current->journal_info = ti;
for (;;) {
up_write(&nilfs->ns_segctor_sem);
current->journal_info = ti->ti_save;
- if (!list_empty(&ti->ti_garbage))
- nilfs_dispose_list(nilfs, &ti->ti_garbage, 0);
}
static void *nilfs_segctor_map_segsum_entry(struct nilfs_sc_info *sci,
}
}
+static void nilfs_iput_work_func(struct work_struct *work)
+{
+ struct nilfs_sc_info *sci = container_of(work, struct nilfs_sc_info,
+ sc_iput_work);
+ struct the_nilfs *nilfs = sci->sc_super->s_fs_info;
+
+ nilfs_dispose_list(nilfs, &sci->sc_iput_queue, 0);
+}
+
static int nilfs_test_metadata_dirty(struct the_nilfs *nilfs,
struct nilfs_root *root)
{
static void nilfs_segctor_drop_written_files(struct nilfs_sc_info *sci,
struct the_nilfs *nilfs)
{
- struct nilfs_transaction_info *ti = current->journal_info;
struct nilfs_inode_info *ii, *n;
+ int defer_iput = false;
spin_lock(&nilfs->ns_inode_lock);
list_for_each_entry_safe(ii, n, &sci->sc_dirty_files, i_dirty) {
clear_bit(NILFS_I_BUSY, &ii->i_state);
brelse(ii->i_bh);
ii->i_bh = NULL;
- list_move_tail(&ii->i_dirty, &ti->ti_garbage);
+ list_del_init(&ii->i_dirty);
+ if (!ii->vfs_inode.i_nlink) {
+ /*
+ * Defer calling iput() to avoid a deadlock
+ * over I_SYNC flag for inodes with i_nlink == 0
+ */
+ list_add_tail(&ii->i_dirty, &sci->sc_iput_queue);
+ defer_iput = true;
+ } else {
+ spin_unlock(&nilfs->ns_inode_lock);
+ iput(&ii->vfs_inode);
+ spin_lock(&nilfs->ns_inode_lock);
+ }
}
spin_unlock(&nilfs->ns_inode_lock);
+
+ if (defer_iput)
+ schedule_work(&sci->sc_iput_work);
}
/*
INIT_LIST_HEAD(&sci->sc_segbufs);
INIT_LIST_HEAD(&sci->sc_write_logs);
INIT_LIST_HEAD(&sci->sc_gc_inodes);
+ INIT_LIST_HEAD(&sci->sc_iput_queue);
+ INIT_WORK(&sci->sc_iput_work, nilfs_iput_work_func);
init_timer(&sci->sc_timer);
sci->sc_interval = HZ * NILFS_SC_DEFAULT_TIMEOUT;
ret = nilfs_segctor_construct(sci, SC_LSEG_SR);
nilfs_transaction_unlock(sci->sc_super);
+ flush_work(&sci->sc_iput_work);
+
} while (ret && retrycount-- > 0);
}
|| sci->sc_seq_request != sci->sc_seq_done);
spin_unlock(&sci->sc_state_lock);
+ if (flush_work(&sci->sc_iput_work))
+ flag = true;
+
if (flag || !nilfs_segctor_confirm(sci))
nilfs_segctor_write_out(sci);
nilfs_dispose_list(nilfs, &sci->sc_dirty_files, 1);
}
+ if (!list_empty(&sci->sc_iput_queue)) {
+ nilfs_warning(sci->sc_super, __func__,
+ "iput queue is not empty\n");
+ nilfs_dispose_list(nilfs, &sci->sc_iput_queue, 1);
+ }
+
WARN_ON(!list_empty(&sci->sc_segbufs));
WARN_ON(!list_empty(&sci->sc_write_logs));
#include <linux/types.h>
#include <linux/fs.h>
#include <linux/buffer_head.h>
+#include <linux/workqueue.h>
#include <linux/nilfs2_fs.h>
#include "nilfs.h"
* @sc_nblk_inc: Block count of current generation
* @sc_dirty_files: List of files to be written
* @sc_gc_inodes: List of GC inodes having blocks to be written
+ * @sc_iput_queue: list of inodes for which iput should be done
+ * @sc_iput_work: work struct to defer iput call
* @sc_freesegs: array of segment numbers to be freed
* @sc_nfreesegs: number of segments on @sc_freesegs
* @sc_dsync_inode: inode whose data pages are written for a sync operation
struct list_head sc_dirty_files;
struct list_head sc_gc_inodes;
+ struct list_head sc_iput_queue;
+ struct work_struct sc_iput_work;
__u64 *sc_freesegs;
size_t sc_nfreesegs;
config FSNOTIFY
def_bool n
+ select SRCU
source "fs/notify/dnotify/Kconfig"
source "fs/notify/inotify/Kconfig"
config QUOTA
bool "Quota support"
select QUOTACTL
+ select SRCU
help
If you say Y here, you will be able to set per user limits for disk
usage (also called disk quotas). Currently, it works for the
}
/* Generic routine for getting common part of quota structure */
-static void do_get_dqblk(struct dquot *dquot, struct fs_disk_quota *di)
+static void do_get_dqblk(struct dquot *dquot, struct qc_dqblk *di)
{
struct mem_dqblk *dm = &dquot->dq_dqb;
memset(di, 0, sizeof(*di));
- di->d_version = FS_DQUOT_VERSION;
- di->d_flags = dquot->dq_id.type == USRQUOTA ?
- FS_USER_QUOTA : FS_GROUP_QUOTA;
- di->d_id = from_kqid_munged(current_user_ns(), dquot->dq_id);
-
spin_lock(&dq_data_lock);
- di->d_blk_hardlimit = stoqb(dm->dqb_bhardlimit);
- di->d_blk_softlimit = stoqb(dm->dqb_bsoftlimit);
+ di->d_spc_hardlimit = dm->dqb_bhardlimit;
+ di->d_spc_softlimit = dm->dqb_bsoftlimit;
di->d_ino_hardlimit = dm->dqb_ihardlimit;
di->d_ino_softlimit = dm->dqb_isoftlimit;
- di->d_bcount = dm->dqb_curspace + dm->dqb_rsvspace;
- di->d_icount = dm->dqb_curinodes;
- di->d_btimer = dm->dqb_btime;
- di->d_itimer = dm->dqb_itime;
+ di->d_space = dm->dqb_curspace + dm->dqb_rsvspace;
+ di->d_ino_count = dm->dqb_curinodes;
+ di->d_spc_timer = dm->dqb_btime;
+ di->d_ino_timer = dm->dqb_itime;
spin_unlock(&dq_data_lock);
}
int dquot_get_dqblk(struct super_block *sb, struct kqid qid,
- struct fs_disk_quota *di)
+ struct qc_dqblk *di)
{
struct dquot *dquot;
}
EXPORT_SYMBOL(dquot_get_dqblk);
-#define VFS_FS_DQ_MASK \
- (FS_DQ_BCOUNT | FS_DQ_BSOFT | FS_DQ_BHARD | \
- FS_DQ_ICOUNT | FS_DQ_ISOFT | FS_DQ_IHARD | \
- FS_DQ_BTIMER | FS_DQ_ITIMER)
+#define VFS_QC_MASK \
+ (QC_SPACE | QC_SPC_SOFT | QC_SPC_HARD | \
+ QC_INO_COUNT | QC_INO_SOFT | QC_INO_HARD | \
+ QC_SPC_TIMER | QC_INO_TIMER)
/* Generic routine for setting common part of quota structure */
-static int do_set_dqblk(struct dquot *dquot, struct fs_disk_quota *di)
+static int do_set_dqblk(struct dquot *dquot, struct qc_dqblk *di)
{
struct mem_dqblk *dm = &dquot->dq_dqb;
int check_blim = 0, check_ilim = 0;
struct mem_dqinfo *dqi = &sb_dqopt(dquot->dq_sb)->info[dquot->dq_id.type];
- if (di->d_fieldmask & ~VFS_FS_DQ_MASK)
+ if (di->d_fieldmask & ~VFS_QC_MASK)
return -EINVAL;
- if (((di->d_fieldmask & FS_DQ_BSOFT) &&
- (di->d_blk_softlimit > dqi->dqi_maxblimit)) ||
- ((di->d_fieldmask & FS_DQ_BHARD) &&
- (di->d_blk_hardlimit > dqi->dqi_maxblimit)) ||
- ((di->d_fieldmask & FS_DQ_ISOFT) &&
+ if (((di->d_fieldmask & QC_SPC_SOFT) &&
+ stoqb(di->d_spc_softlimit) > dqi->dqi_maxblimit) ||
+ ((di->d_fieldmask & QC_SPC_HARD) &&
+ stoqb(di->d_spc_hardlimit) > dqi->dqi_maxblimit) ||
+ ((di->d_fieldmask & QC_INO_SOFT) &&
(di->d_ino_softlimit > dqi->dqi_maxilimit)) ||
- ((di->d_fieldmask & FS_DQ_IHARD) &&
+ ((di->d_fieldmask & QC_INO_HARD) &&
(di->d_ino_hardlimit > dqi->dqi_maxilimit)))
return -ERANGE;
spin_lock(&dq_data_lock);
- if (di->d_fieldmask & FS_DQ_BCOUNT) {
- dm->dqb_curspace = di->d_bcount - dm->dqb_rsvspace;
+ if (di->d_fieldmask & QC_SPACE) {
+ dm->dqb_curspace = di->d_space - dm->dqb_rsvspace;
check_blim = 1;
set_bit(DQ_LASTSET_B + QIF_SPACE_B, &dquot->dq_flags);
}
- if (di->d_fieldmask & FS_DQ_BSOFT)
- dm->dqb_bsoftlimit = qbtos(di->d_blk_softlimit);
- if (di->d_fieldmask & FS_DQ_BHARD)
- dm->dqb_bhardlimit = qbtos(di->d_blk_hardlimit);
- if (di->d_fieldmask & (FS_DQ_BSOFT | FS_DQ_BHARD)) {
+ if (di->d_fieldmask & QC_SPC_SOFT)
+ dm->dqb_bsoftlimit = di->d_spc_softlimit;
+ if (di->d_fieldmask & QC_SPC_HARD)
+ dm->dqb_bhardlimit = di->d_spc_hardlimit;
+ if (di->d_fieldmask & (QC_SPC_SOFT | QC_SPC_HARD)) {
check_blim = 1;
set_bit(DQ_LASTSET_B + QIF_BLIMITS_B, &dquot->dq_flags);
}
- if (di->d_fieldmask & FS_DQ_ICOUNT) {
- dm->dqb_curinodes = di->d_icount;
+ if (di->d_fieldmask & QC_INO_COUNT) {
+ dm->dqb_curinodes = di->d_ino_count;
check_ilim = 1;
set_bit(DQ_LASTSET_B + QIF_INODES_B, &dquot->dq_flags);
}
- if (di->d_fieldmask & FS_DQ_ISOFT)
+ if (di->d_fieldmask & QC_INO_SOFT)
dm->dqb_isoftlimit = di->d_ino_softlimit;
- if (di->d_fieldmask & FS_DQ_IHARD)
+ if (di->d_fieldmask & QC_INO_HARD)
dm->dqb_ihardlimit = di->d_ino_hardlimit;
- if (di->d_fieldmask & (FS_DQ_ISOFT | FS_DQ_IHARD)) {
+ if (di->d_fieldmask & (QC_INO_SOFT | QC_INO_HARD)) {
check_ilim = 1;
set_bit(DQ_LASTSET_B + QIF_ILIMITS_B, &dquot->dq_flags);
}
- if (di->d_fieldmask & FS_DQ_BTIMER) {
- dm->dqb_btime = di->d_btimer;
+ if (di->d_fieldmask & QC_SPC_TIMER) {
+ dm->dqb_btime = di->d_spc_timer;
check_blim = 1;
set_bit(DQ_LASTSET_B + QIF_BTIME_B, &dquot->dq_flags);
}
- if (di->d_fieldmask & FS_DQ_ITIMER) {
- dm->dqb_itime = di->d_itimer;
+ if (di->d_fieldmask & QC_INO_TIMER) {
+ dm->dqb_itime = di->d_ino_timer;
check_ilim = 1;
set_bit(DQ_LASTSET_B + QIF_ITIME_B, &dquot->dq_flags);
}
dm->dqb_curspace < dm->dqb_bsoftlimit) {
dm->dqb_btime = 0;
clear_bit(DQ_BLKS_B, &dquot->dq_flags);
- } else if (!(di->d_fieldmask & FS_DQ_BTIMER))
+ } else if (!(di->d_fieldmask & QC_SPC_TIMER))
/* Set grace only if user hasn't provided his own... */
dm->dqb_btime = get_seconds() + dqi->dqi_bgrace;
}
dm->dqb_curinodes < dm->dqb_isoftlimit) {
dm->dqb_itime = 0;
clear_bit(DQ_INODES_B, &dquot->dq_flags);
- } else if (!(di->d_fieldmask & FS_DQ_ITIMER))
+ } else if (!(di->d_fieldmask & QC_INO_TIMER))
/* Set grace only if user hasn't provided his own... */
dm->dqb_itime = get_seconds() + dqi->dqi_igrace;
}
}
int dquot_set_dqblk(struct super_block *sb, struct kqid qid,
- struct fs_disk_quota *di)
+ struct qc_dqblk *di)
{
struct dquot *dquot;
int rc;
return sb->s_qcop->set_info(sb, type, &info);
}
-static void copy_to_if_dqblk(struct if_dqblk *dst, struct fs_disk_quota *src)
+static inline qsize_t qbtos(qsize_t blocks)
+{
+ return blocks << QIF_DQBLKSIZE_BITS;
+}
+
+static inline qsize_t stoqb(qsize_t space)
+{
+ return (space + QIF_DQBLKSIZE - 1) >> QIF_DQBLKSIZE_BITS;
+}
+
+static void copy_to_if_dqblk(struct if_dqblk *dst, struct qc_dqblk *src)
{
memset(dst, 0, sizeof(*dst));
- dst->dqb_bhardlimit = src->d_blk_hardlimit;
- dst->dqb_bsoftlimit = src->d_blk_softlimit;
- dst->dqb_curspace = src->d_bcount;
+ dst->dqb_bhardlimit = stoqb(src->d_spc_hardlimit);
+ dst->dqb_bsoftlimit = stoqb(src->d_spc_softlimit);
+ dst->dqb_curspace = src->d_space;
dst->dqb_ihardlimit = src->d_ino_hardlimit;
dst->dqb_isoftlimit = src->d_ino_softlimit;
- dst->dqb_curinodes = src->d_icount;
- dst->dqb_btime = src->d_btimer;
- dst->dqb_itime = src->d_itimer;
+ dst->dqb_curinodes = src->d_ino_count;
+ dst->dqb_btime = src->d_spc_timer;
+ dst->dqb_itime = src->d_ino_timer;
dst->dqb_valid = QIF_ALL;
}
void __user *addr)
{
struct kqid qid;
- struct fs_disk_quota fdq;
+ struct qc_dqblk fdq;
struct if_dqblk idq;
int ret;
return 0;
}
-static void copy_from_if_dqblk(struct fs_disk_quota *dst, struct if_dqblk *src)
+static void copy_from_if_dqblk(struct qc_dqblk *dst, struct if_dqblk *src)
{
- dst->d_blk_hardlimit = src->dqb_bhardlimit;
- dst->d_blk_softlimit = src->dqb_bsoftlimit;
- dst->d_bcount = src->dqb_curspace;
+ dst->d_spc_hardlimit = qbtos(src->dqb_bhardlimit);
+ dst->d_spc_softlimit = qbtos(src->dqb_bsoftlimit);
+ dst->d_space = src->dqb_curspace;
dst->d_ino_hardlimit = src->dqb_ihardlimit;
dst->d_ino_softlimit = src->dqb_isoftlimit;
- dst->d_icount = src->dqb_curinodes;
- dst->d_btimer = src->dqb_btime;
- dst->d_itimer = src->dqb_itime;
+ dst->d_ino_count = src->dqb_curinodes;
+ dst->d_spc_timer = src->dqb_btime;
+ dst->d_ino_timer = src->dqb_itime;
dst->d_fieldmask = 0;
if (src->dqb_valid & QIF_BLIMITS)
- dst->d_fieldmask |= FS_DQ_BSOFT | FS_DQ_BHARD;
+ dst->d_fieldmask |= QC_SPC_SOFT | QC_SPC_HARD;
if (src->dqb_valid & QIF_SPACE)
- dst->d_fieldmask |= FS_DQ_BCOUNT;
+ dst->d_fieldmask |= QC_SPACE;
if (src->dqb_valid & QIF_ILIMITS)
- dst->d_fieldmask |= FS_DQ_ISOFT | FS_DQ_IHARD;
+ dst->d_fieldmask |= QC_INO_SOFT | QC_INO_HARD;
if (src->dqb_valid & QIF_INODES)
- dst->d_fieldmask |= FS_DQ_ICOUNT;
+ dst->d_fieldmask |= QC_INO_COUNT;
if (src->dqb_valid & QIF_BTIME)
- dst->d_fieldmask |= FS_DQ_BTIMER;
+ dst->d_fieldmask |= QC_SPC_TIMER;
if (src->dqb_valid & QIF_ITIME)
- dst->d_fieldmask |= FS_DQ_ITIMER;
+ dst->d_fieldmask |= QC_INO_TIMER;
}
static int quota_setquota(struct super_block *sb, int type, qid_t id,
void __user *addr)
{
- struct fs_disk_quota fdq;
+ struct qc_dqblk fdq;
struct if_dqblk idq;
struct kqid qid;
return ret;
}
+/*
+ * XFS defines BBTOB and BTOBB macros inside fs/xfs/ and we cannot move them
+ * out of there as xfsprogs rely on definitions being in that header file. So
+ * just define same functions here for quota purposes.
+ */
+#define XFS_BB_SHIFT 9
+
+static inline u64 quota_bbtob(u64 blocks)
+{
+ return blocks << XFS_BB_SHIFT;
+}
+
+static inline u64 quota_btobb(u64 bytes)
+{
+ return (bytes + (1 << XFS_BB_SHIFT) - 1) >> XFS_BB_SHIFT;
+}
+
+static void copy_from_xfs_dqblk(struct qc_dqblk *dst, struct fs_disk_quota *src)
+{
+ dst->d_spc_hardlimit = quota_bbtob(src->d_blk_hardlimit);
+ dst->d_spc_softlimit = quota_bbtob(src->d_blk_softlimit);
+ dst->d_ino_hardlimit = src->d_ino_hardlimit;
+ dst->d_ino_softlimit = src->d_ino_softlimit;
+ dst->d_space = quota_bbtob(src->d_bcount);
+ dst->d_ino_count = src->d_icount;
+ dst->d_ino_timer = src->d_itimer;
+ dst->d_spc_timer = src->d_btimer;
+ dst->d_ino_warns = src->d_iwarns;
+ dst->d_spc_warns = src->d_bwarns;
+ dst->d_rt_spc_hardlimit = quota_bbtob(src->d_rtb_hardlimit);
+ dst->d_rt_spc_softlimit = quota_bbtob(src->d_rtb_softlimit);
+ dst->d_rt_space = quota_bbtob(src->d_rtbcount);
+ dst->d_rt_spc_timer = src->d_rtbtimer;
+ dst->d_rt_spc_warns = src->d_rtbwarns;
+ dst->d_fieldmask = 0;
+ if (src->d_fieldmask & FS_DQ_ISOFT)
+ dst->d_fieldmask |= QC_INO_SOFT;
+ if (src->d_fieldmask & FS_DQ_IHARD)
+ dst->d_fieldmask |= QC_INO_HARD;
+ if (src->d_fieldmask & FS_DQ_BSOFT)
+ dst->d_fieldmask |= QC_SPC_SOFT;
+ if (src->d_fieldmask & FS_DQ_BHARD)
+ dst->d_fieldmask |= QC_SPC_HARD;
+ if (src->d_fieldmask & FS_DQ_RTBSOFT)
+ dst->d_fieldmask |= QC_RT_SPC_SOFT;
+ if (src->d_fieldmask & FS_DQ_RTBHARD)
+ dst->d_fieldmask |= QC_RT_SPC_HARD;
+ if (src->d_fieldmask & FS_DQ_BTIMER)
+ dst->d_fieldmask |= QC_SPC_TIMER;
+ if (src->d_fieldmask & FS_DQ_ITIMER)
+ dst->d_fieldmask |= QC_INO_TIMER;
+ if (src->d_fieldmask & FS_DQ_RTBTIMER)
+ dst->d_fieldmask |= QC_RT_SPC_TIMER;
+ if (src->d_fieldmask & FS_DQ_BWARNS)
+ dst->d_fieldmask |= QC_SPC_WARNS;
+ if (src->d_fieldmask & FS_DQ_IWARNS)
+ dst->d_fieldmask |= QC_INO_WARNS;
+ if (src->d_fieldmask & FS_DQ_RTBWARNS)
+ dst->d_fieldmask |= QC_RT_SPC_WARNS;
+ if (src->d_fieldmask & FS_DQ_BCOUNT)
+ dst->d_fieldmask |= QC_SPACE;
+ if (src->d_fieldmask & FS_DQ_ICOUNT)
+ dst->d_fieldmask |= QC_INO_COUNT;
+ if (src->d_fieldmask & FS_DQ_RTBCOUNT)
+ dst->d_fieldmask |= QC_RT_SPACE;
+}
+
static int quota_setxquota(struct super_block *sb, int type, qid_t id,
void __user *addr)
{
struct fs_disk_quota fdq;
+ struct qc_dqblk qdq;
struct kqid qid;
if (copy_from_user(&fdq, addr, sizeof(fdq)))
qid = make_kqid(current_user_ns(), type, id);
if (!qid_valid(qid))
return -EINVAL;
- return sb->s_qcop->set_dqblk(sb, qid, &fdq);
+ copy_from_xfs_dqblk(&qdq, &fdq);
+ return sb->s_qcop->set_dqblk(sb, qid, &qdq);
+}
+
+static void copy_to_xfs_dqblk(struct fs_disk_quota *dst, struct qc_dqblk *src,
+ int type, qid_t id)
+{
+ memset(dst, 0, sizeof(*dst));
+ dst->d_version = FS_DQUOT_VERSION;
+ dst->d_id = id;
+ if (type == USRQUOTA)
+ dst->d_flags = FS_USER_QUOTA;
+ else if (type == PRJQUOTA)
+ dst->d_flags = FS_PROJ_QUOTA;
+ else
+ dst->d_flags = FS_GROUP_QUOTA;
+ dst->d_blk_hardlimit = quota_btobb(src->d_spc_hardlimit);
+ dst->d_blk_softlimit = quota_btobb(src->d_spc_softlimit);
+ dst->d_ino_hardlimit = src->d_ino_hardlimit;
+ dst->d_ino_softlimit = src->d_ino_softlimit;
+ dst->d_bcount = quota_btobb(src->d_space);
+ dst->d_icount = src->d_ino_count;
+ dst->d_itimer = src->d_ino_timer;
+ dst->d_btimer = src->d_spc_timer;
+ dst->d_iwarns = src->d_ino_warns;
+ dst->d_bwarns = src->d_spc_warns;
+ dst->d_rtb_hardlimit = quota_btobb(src->d_rt_spc_hardlimit);
+ dst->d_rtb_softlimit = quota_btobb(src->d_rt_spc_softlimit);
+ dst->d_rtbcount = quota_btobb(src->d_rt_space);
+ dst->d_rtbtimer = src->d_rt_spc_timer;
+ dst->d_rtbwarns = src->d_rt_spc_warns;
}
static int quota_getxquota(struct super_block *sb, int type, qid_t id,
void __user *addr)
{
struct fs_disk_quota fdq;
+ struct qc_dqblk qdq;
struct kqid qid;
int ret;
qid = make_kqid(current_user_ns(), type, id);
if (!qid_valid(qid))
return -EINVAL;
- ret = sb->s_qcop->get_dqblk(sb, qid, &fdq);
- if (!ret && copy_to_user(addr, &fdq, sizeof(fdq)))
+ ret = sb->s_qcop->get_dqblk(sb, qid, &qdq);
+ if (ret)
+ return ret;
+ copy_to_xfs_dqblk(&fdq, &qdq, type, id);
+ if (copy_to_user(addr, &fdq, sizeof(fdq)))
return -EFAULT;
return ret;
}
static int udf_release_file(struct inode *inode, struct file *filp)
{
if (filp->f_mode & FMODE_WRITE &&
- atomic_read(&inode->i_writecount) > 1) {
+ atomic_read(&inode->i_writecount) == 1) {
/*
* Grab i_mutex to avoid races with writes changing i_size
* while we are running.
/* quota ops */
extern int xfs_qm_scall_trunc_qfiles(struct xfs_mount *, uint);
extern int xfs_qm_scall_getquota(struct xfs_mount *, xfs_dqid_t,
- uint, struct fs_disk_quota *);
+ uint, struct qc_dqblk *);
extern int xfs_qm_scall_setqlim(struct xfs_mount *, xfs_dqid_t, uint,
- struct fs_disk_quota *);
+ struct qc_dqblk *);
extern int xfs_qm_scall_getqstat(struct xfs_mount *,
struct fs_quota_stat *);
extern int xfs_qm_scall_getqstatv(struct xfs_mount *,
STATIC int xfs_qm_log_quotaoff_end(xfs_mount_t *, xfs_qoff_logitem_t *,
uint);
STATIC uint xfs_qm_export_flags(uint);
-STATIC uint xfs_qm_export_qtype_flags(uint);
/*
* Turn off quota accounting and/or enforcement for all udquots and/or
return 0;
}
-#define XFS_DQ_MASK \
- (FS_DQ_LIMIT_MASK | FS_DQ_TIMER_MASK | FS_DQ_WARNS_MASK)
+#define XFS_QC_MASK \
+ (QC_LIMIT_MASK | QC_TIMER_MASK | QC_WARNS_MASK)
/*
* Adjust quota limits, and start/stop timers accordingly.
struct xfs_mount *mp,
xfs_dqid_t id,
uint type,
- fs_disk_quota_t *newlim)
+ struct qc_dqblk *newlim)
{
struct xfs_quotainfo *q = mp->m_quotainfo;
struct xfs_disk_dquot *ddq;
int error;
xfs_qcnt_t hard, soft;
- if (newlim->d_fieldmask & ~XFS_DQ_MASK)
+ if (newlim->d_fieldmask & ~XFS_QC_MASK)
return -EINVAL;
- if ((newlim->d_fieldmask & XFS_DQ_MASK) == 0)
+ if ((newlim->d_fieldmask & XFS_QC_MASK) == 0)
return 0;
/*
/*
* Make sure that hardlimits are >= soft limits before changing.
*/
- hard = (newlim->d_fieldmask & FS_DQ_BHARD) ?
- (xfs_qcnt_t) XFS_BB_TO_FSB(mp, newlim->d_blk_hardlimit) :
+ hard = (newlim->d_fieldmask & QC_SPC_HARD) ?
+ (xfs_qcnt_t) XFS_B_TO_FSB(mp, newlim->d_spc_hardlimit) :
be64_to_cpu(ddq->d_blk_hardlimit);
- soft = (newlim->d_fieldmask & FS_DQ_BSOFT) ?
- (xfs_qcnt_t) XFS_BB_TO_FSB(mp, newlim->d_blk_softlimit) :
+ soft = (newlim->d_fieldmask & QC_SPC_SOFT) ?
+ (xfs_qcnt_t) XFS_B_TO_FSB(mp, newlim->d_spc_softlimit) :
be64_to_cpu(ddq->d_blk_softlimit);
if (hard == 0 || hard >= soft) {
ddq->d_blk_hardlimit = cpu_to_be64(hard);
} else {
xfs_debug(mp, "blkhard %Ld < blksoft %Ld", hard, soft);
}
- hard = (newlim->d_fieldmask & FS_DQ_RTBHARD) ?
- (xfs_qcnt_t) XFS_BB_TO_FSB(mp, newlim->d_rtb_hardlimit) :
+ hard = (newlim->d_fieldmask & QC_RT_SPC_HARD) ?
+ (xfs_qcnt_t) XFS_B_TO_FSB(mp, newlim->d_rt_spc_hardlimit) :
be64_to_cpu(ddq->d_rtb_hardlimit);
- soft = (newlim->d_fieldmask & FS_DQ_RTBSOFT) ?
- (xfs_qcnt_t) XFS_BB_TO_FSB(mp, newlim->d_rtb_softlimit) :
+ soft = (newlim->d_fieldmask & QC_RT_SPC_SOFT) ?
+ (xfs_qcnt_t) XFS_B_TO_FSB(mp, newlim->d_rt_spc_softlimit) :
be64_to_cpu(ddq->d_rtb_softlimit);
if (hard == 0 || hard >= soft) {
ddq->d_rtb_hardlimit = cpu_to_be64(hard);
xfs_debug(mp, "rtbhard %Ld < rtbsoft %Ld", hard, soft);
}
- hard = (newlim->d_fieldmask & FS_DQ_IHARD) ?
+ hard = (newlim->d_fieldmask & QC_INO_HARD) ?
(xfs_qcnt_t) newlim->d_ino_hardlimit :
be64_to_cpu(ddq->d_ino_hardlimit);
- soft = (newlim->d_fieldmask & FS_DQ_ISOFT) ?
+ soft = (newlim->d_fieldmask & QC_INO_SOFT) ?
(xfs_qcnt_t) newlim->d_ino_softlimit :
be64_to_cpu(ddq->d_ino_softlimit);
if (hard == 0 || hard >= soft) {
/*
* Update warnings counter(s) if requested
*/
- if (newlim->d_fieldmask & FS_DQ_BWARNS)
- ddq->d_bwarns = cpu_to_be16(newlim->d_bwarns);
- if (newlim->d_fieldmask & FS_DQ_IWARNS)
- ddq->d_iwarns = cpu_to_be16(newlim->d_iwarns);
- if (newlim->d_fieldmask & FS_DQ_RTBWARNS)
- ddq->d_rtbwarns = cpu_to_be16(newlim->d_rtbwarns);
+ if (newlim->d_fieldmask & QC_SPC_WARNS)
+ ddq->d_bwarns = cpu_to_be16(newlim->d_spc_warns);
+ if (newlim->d_fieldmask & QC_INO_WARNS)
+ ddq->d_iwarns = cpu_to_be16(newlim->d_ino_warns);
+ if (newlim->d_fieldmask & QC_RT_SPC_WARNS)
+ ddq->d_rtbwarns = cpu_to_be16(newlim->d_rt_spc_warns);
if (id == 0) {
/*
* soft and hard limit values (already done, above), and
* for warnings.
*/
- if (newlim->d_fieldmask & FS_DQ_BTIMER) {
- q->qi_btimelimit = newlim->d_btimer;
- ddq->d_btimer = cpu_to_be32(newlim->d_btimer);
+ if (newlim->d_fieldmask & QC_SPC_TIMER) {
+ q->qi_btimelimit = newlim->d_spc_timer;
+ ddq->d_btimer = cpu_to_be32(newlim->d_spc_timer);
}
- if (newlim->d_fieldmask & FS_DQ_ITIMER) {
- q->qi_itimelimit = newlim->d_itimer;
- ddq->d_itimer = cpu_to_be32(newlim->d_itimer);
+ if (newlim->d_fieldmask & QC_INO_TIMER) {
+ q->qi_itimelimit = newlim->d_ino_timer;
+ ddq->d_itimer = cpu_to_be32(newlim->d_ino_timer);
}
- if (newlim->d_fieldmask & FS_DQ_RTBTIMER) {
- q->qi_rtbtimelimit = newlim->d_rtbtimer;
- ddq->d_rtbtimer = cpu_to_be32(newlim->d_rtbtimer);
+ if (newlim->d_fieldmask & QC_RT_SPC_TIMER) {
+ q->qi_rtbtimelimit = newlim->d_rt_spc_timer;
+ ddq->d_rtbtimer = cpu_to_be32(newlim->d_rt_spc_timer);
}
- if (newlim->d_fieldmask & FS_DQ_BWARNS)
- q->qi_bwarnlimit = newlim->d_bwarns;
- if (newlim->d_fieldmask & FS_DQ_IWARNS)
- q->qi_iwarnlimit = newlim->d_iwarns;
- if (newlim->d_fieldmask & FS_DQ_RTBWARNS)
- q->qi_rtbwarnlimit = newlim->d_rtbwarns;
+ if (newlim->d_fieldmask & QC_SPC_WARNS)
+ q->qi_bwarnlimit = newlim->d_spc_warns;
+ if (newlim->d_fieldmask & QC_INO_WARNS)
+ q->qi_iwarnlimit = newlim->d_ino_warns;
+ if (newlim->d_fieldmask & QC_RT_SPC_WARNS)
+ q->qi_rtbwarnlimit = newlim->d_rt_spc_warns;
} else {
/*
* If the user is now over quota, start the timelimit.
struct xfs_mount *mp,
xfs_dqid_t id,
uint type,
- struct fs_disk_quota *dst)
+ struct qc_dqblk *dst)
{
struct xfs_dquot *dqp;
int error;
}
memset(dst, 0, sizeof(*dst));
- dst->d_version = FS_DQUOT_VERSION;
- dst->d_flags = xfs_qm_export_qtype_flags(dqp->q_core.d_flags);
- dst->d_id = be32_to_cpu(dqp->q_core.d_id);
- dst->d_blk_hardlimit =
- XFS_FSB_TO_BB(mp, be64_to_cpu(dqp->q_core.d_blk_hardlimit));
- dst->d_blk_softlimit =
- XFS_FSB_TO_BB(mp, be64_to_cpu(dqp->q_core.d_blk_softlimit));
+ dst->d_spc_hardlimit =
+ XFS_FSB_TO_B(mp, be64_to_cpu(dqp->q_core.d_blk_hardlimit));
+ dst->d_spc_softlimit =
+ XFS_FSB_TO_B(mp, be64_to_cpu(dqp->q_core.d_blk_softlimit));
dst->d_ino_hardlimit = be64_to_cpu(dqp->q_core.d_ino_hardlimit);
dst->d_ino_softlimit = be64_to_cpu(dqp->q_core.d_ino_softlimit);
- dst->d_bcount = XFS_FSB_TO_BB(mp, dqp->q_res_bcount);
- dst->d_icount = dqp->q_res_icount;
- dst->d_btimer = be32_to_cpu(dqp->q_core.d_btimer);
- dst->d_itimer = be32_to_cpu(dqp->q_core.d_itimer);
- dst->d_iwarns = be16_to_cpu(dqp->q_core.d_iwarns);
- dst->d_bwarns = be16_to_cpu(dqp->q_core.d_bwarns);
- dst->d_rtb_hardlimit =
- XFS_FSB_TO_BB(mp, be64_to_cpu(dqp->q_core.d_rtb_hardlimit));
- dst->d_rtb_softlimit =
- XFS_FSB_TO_BB(mp, be64_to_cpu(dqp->q_core.d_rtb_softlimit));
- dst->d_rtbcount = XFS_FSB_TO_BB(mp, dqp->q_res_rtbcount);
- dst->d_rtbtimer = be32_to_cpu(dqp->q_core.d_rtbtimer);
- dst->d_rtbwarns = be16_to_cpu(dqp->q_core.d_rtbwarns);
+ dst->d_space = XFS_FSB_TO_B(mp, dqp->q_res_bcount);
+ dst->d_ino_count = dqp->q_res_icount;
+ dst->d_spc_timer = be32_to_cpu(dqp->q_core.d_btimer);
+ dst->d_ino_timer = be32_to_cpu(dqp->q_core.d_itimer);
+ dst->d_ino_warns = be16_to_cpu(dqp->q_core.d_iwarns);
+ dst->d_spc_warns = be16_to_cpu(dqp->q_core.d_bwarns);
+ dst->d_rt_spc_hardlimit =
+ XFS_FSB_TO_B(mp, be64_to_cpu(dqp->q_core.d_rtb_hardlimit));
+ dst->d_rt_spc_softlimit =
+ XFS_FSB_TO_B(mp, be64_to_cpu(dqp->q_core.d_rtb_softlimit));
+ dst->d_rt_space = XFS_FSB_TO_B(mp, dqp->q_res_rtbcount);
+ dst->d_rt_spc_timer = be32_to_cpu(dqp->q_core.d_rtbtimer);
+ dst->d_rt_spc_warns = be16_to_cpu(dqp->q_core.d_rtbwarns);
/*
* Internally, we don't reset all the timers when quota enforcement
dqp->q_core.d_flags == XFS_DQ_GROUP) ||
(!XFS_IS_PQUOTA_ENFORCED(mp) &&
dqp->q_core.d_flags == XFS_DQ_PROJ)) {
- dst->d_btimer = 0;
- dst->d_itimer = 0;
- dst->d_rtbtimer = 0;
+ dst->d_spc_timer = 0;
+ dst->d_ino_timer = 0;
+ dst->d_rt_spc_timer = 0;
}
#ifdef DEBUG
- if (((XFS_IS_UQUOTA_ENFORCED(mp) && dst->d_flags == FS_USER_QUOTA) ||
- (XFS_IS_GQUOTA_ENFORCED(mp) && dst->d_flags == FS_GROUP_QUOTA) ||
- (XFS_IS_PQUOTA_ENFORCED(mp) && dst->d_flags == FS_PROJ_QUOTA)) &&
- dst->d_id != 0) {
- if ((dst->d_bcount > dst->d_blk_softlimit) &&
- (dst->d_blk_softlimit > 0)) {
- ASSERT(dst->d_btimer != 0);
+ if (((XFS_IS_UQUOTA_ENFORCED(mp) && type == XFS_DQ_USER) ||
+ (XFS_IS_GQUOTA_ENFORCED(mp) && type == XFS_DQ_GROUP) ||
+ (XFS_IS_PQUOTA_ENFORCED(mp) && type == XFS_DQ_PROJ)) &&
+ id != 0) {
+ if ((dst->d_space > dst->d_spc_softlimit) &&
+ (dst->d_spc_softlimit > 0)) {
+ ASSERT(dst->d_spc_timer != 0);
}
- if ((dst->d_icount > dst->d_ino_softlimit) &&
+ if ((dst->d_ino_count > dst->d_ino_softlimit) &&
(dst->d_ino_softlimit > 0)) {
- ASSERT(dst->d_itimer != 0);
+ ASSERT(dst->d_ino_timer != 0);
}
}
#endif
return error;
}
-STATIC uint
-xfs_qm_export_qtype_flags(
- uint flags)
-{
- /*
- * Can't be more than one, or none.
- */
- ASSERT((flags & (FS_PROJ_QUOTA | FS_USER_QUOTA)) !=
- (FS_PROJ_QUOTA | FS_USER_QUOTA));
- ASSERT((flags & (FS_PROJ_QUOTA | FS_GROUP_QUOTA)) !=
- (FS_PROJ_QUOTA | FS_GROUP_QUOTA));
- ASSERT((flags & (FS_USER_QUOTA | FS_GROUP_QUOTA)) !=
- (FS_USER_QUOTA | FS_GROUP_QUOTA));
- ASSERT((flags & (FS_PROJ_QUOTA|FS_USER_QUOTA|FS_GROUP_QUOTA)) != 0);
-
- return (flags & XFS_DQ_USER) ?
- FS_USER_QUOTA : (flags & XFS_DQ_PROJ) ?
- FS_PROJ_QUOTA : FS_GROUP_QUOTA;
-}
-
STATIC uint
xfs_qm_export_flags(
uint flags)
xfs_fs_get_dqblk(
struct super_block *sb,
struct kqid qid,
- struct fs_disk_quota *fdq)
+ struct qc_dqblk *qdq)
{
struct xfs_mount *mp = XFS_M(sb);
return -ESRCH;
return xfs_qm_scall_getquota(mp, from_kqid(&init_user_ns, qid),
- xfs_quota_type(qid.type), fdq);
+ xfs_quota_type(qid.type), qdq);
}
STATIC int
xfs_fs_set_dqblk(
struct super_block *sb,
struct kqid qid,
- struct fs_disk_quota *fdq)
+ struct qc_dqblk *qdq)
{
struct xfs_mount *mp = XFS_M(sb);
return -ESRCH;
return xfs_qm_scall_setqlim(mp, from_kqid(&init_user_ns, qid),
- xfs_quota_type(qid.type), fdq);
+ xfs_quota_type(qid.type), qdq);
}
const struct quotactl_ops xfs_quotactl_operations = {
/* Is this type a native word size -- useful for atomic operations */
#ifndef __native_word
-# define __native_word(t) (sizeof(t) == sizeof(int) || sizeof(t) == sizeof(long))
+# define __native_word(t) (sizeof(t) == sizeof(char) || sizeof(t) == sizeof(short) || sizeof(t) == sizeof(int) || sizeof(t) == sizeof(long))
#endif
/* Compile time object size, -1 for unknown */
char *filter_str);
extern void ftrace_profile_free_filter(struct perf_event *event);
extern void *perf_trace_buf_prepare(int size, unsigned short type,
- struct pt_regs *regs, int *rctxp);
+ struct pt_regs **regs, int *rctxp);
static inline void
perf_trace_buf_submit(void *raw_data, int size, int rctx, u64 addr,
* @clock_was_set: Indicates that clock was set from irq context.
* @expires_next: absolute time of the next event which was scheduled
* via clock_set_next_event()
+ * @in_hrtirq: hrtimer_interrupt() is currently executing
* @hres_active: State of high resolution mode
* @hang_detected: The last hrtimer interrupt detected a hang
* @nr_events: Total number of hrtimer interrupt events
unsigned int clock_was_set;
#ifdef CONFIG_HIGH_RES_TIMERS
ktime_t expires_next;
+ int in_hrtirq;
int hres_active;
int hang_detected;
unsigned long nr_events;
struct device dev; /* the device structure */
int irq; /* irq issued by device */
struct list_head detected;
+#if IS_ENABLED(CONFIG_I2C_SLAVE)
i2c_slave_cb_t slave_cb; /* callback for slave mode */
+#endif
};
#define to_i2c_client(d) container_of(d, struct i2c_client, dev)
/* I2C slave support */
+#if IS_ENABLED(CONFIG_I2C_SLAVE)
enum i2c_slave_event {
I2C_SLAVE_REQ_READ_START,
I2C_SLAVE_REQ_READ_END,
{
return client->slave_cb(client, event, val);
}
+#endif
/**
* struct i2c_board_info - template for device creation
/* To determine what the adapter supports */
u32 (*functionality) (struct i2c_adapter *);
+#if IS_ENABLED(CONFIG_I2C_SLAVE)
int (*reg_slave)(struct i2c_client *client);
int (*unreg_slave)(struct i2c_client *client);
+#endif
};
/**
/**
* vlan_get_protocol - get protocol EtherType.
* @skb: skbuff to query
+ * @type: first vlan protocol
+ * @depth: buffer to store length of eth and vlan tags in bytes
*
* Returns the EtherType of the packet, regardless of whether it is
* vlan encapsulated (normal or hardware accelerated) or not.
*/
-static inline __be16 vlan_get_protocol(const struct sk_buff *skb)
+static inline __be16 __vlan_get_protocol(struct sk_buff *skb, __be16 type,
+ int *depth)
{
- __be16 protocol = 0;
-
- if (vlan_tx_tag_present(skb) ||
- skb->protocol != cpu_to_be16(ETH_P_8021Q))
- protocol = skb->protocol;
- else {
- __be16 proto, *protop;
- protop = skb_header_pointer(skb, offsetof(struct vlan_ethhdr,
- h_vlan_encapsulated_proto),
- sizeof(proto), &proto);
- if (likely(protop))
- protocol = *protop;
+ unsigned int vlan_depth = skb->mac_len;
+
+ /* if type is 802.1Q/AD then the header should already be
+ * present at mac_len - VLAN_HLEN (if mac_len > 0), or at
+ * ETH_HLEN otherwise
+ */
+ if (type == htons(ETH_P_8021Q) || type == htons(ETH_P_8021AD)) {
+ if (vlan_depth) {
+ if (WARN_ON(vlan_depth < VLAN_HLEN))
+ return 0;
+ vlan_depth -= VLAN_HLEN;
+ } else {
+ vlan_depth = ETH_HLEN;
+ }
+ do {
+ struct vlan_hdr *vh;
+
+ if (unlikely(!pskb_may_pull(skb,
+ vlan_depth + VLAN_HLEN)))
+ return 0;
+
+ vh = (struct vlan_hdr *)(skb->data + vlan_depth);
+ type = vh->h_vlan_encapsulated_proto;
+ vlan_depth += VLAN_HLEN;
+ } while (type == htons(ETH_P_8021Q) ||
+ type == htons(ETH_P_8021AD));
}
- return protocol;
+ if (depth)
+ *depth = vlan_depth;
+
+ return type;
+}
+
+/**
+ * vlan_get_protocol - get protocol EtherType.
+ * @skb: skbuff to query
+ *
+ * Returns the EtherType of the packet, regardless of whether it is
+ * vlan encapsulated (normal or hardware accelerated) or not.
+ */
+static inline __be16 vlan_get_protocol(struct sk_buff *skb)
+{
+ return __vlan_get_protocol(skb, skb->protocol, NULL);
}
static inline void vlan_set_encap_proto(struct sk_buff *skb,
*/
# define might_sleep() \
do { __might_sleep(__FILE__, __LINE__, 0); might_resched(); } while (0)
-# define sched_annotate_sleep() __set_current_state(TASK_RUNNING)
+# define sched_annotate_sleep() (current->task_state_change = 0)
#else
static inline void ___might_sleep(const char *file, int line,
int preempt_offset) { }
}
#if BITS_PER_LONG < 64
-extern u64 ktime_divns(const ktime_t kt, s64 div);
+extern u64 __ktime_divns(const ktime_t kt, s64 div);
+static inline u64 ktime_divns(const ktime_t kt, s64 div)
+{
+ if (__builtin_constant_p(div) && !(div >> 32)) {
+ u64 ns = kt.tv64;
+ do_div(ns, div);
+ return ns;
+ } else {
+ return __ktime_divns(kt, div);
+ }
+}
#else /* BITS_PER_LONG < 64 */
# define ktime_divns(kt, div) (u64)((kt).tv64 / (div))
#endif
return ktime_to_us(ktime_sub(later, earlier));
}
+static inline s64 ktime_ms_delta(const ktime_t later, const ktime_t earlier)
+{
+ return ktime_to_ms(ktime_sub(later, earlier));
+}
+
static inline ktime_t ktime_add_us(const ktime_t kt, const u64 usec)
{
return ktime_add_ns(kt, usec * NSEC_PER_USEC);
MLX4_MAX_NUM_PF = 16,
MLX4_MAX_NUM_VF = 126,
MLX4_MAX_NUM_VF_P_PORT = 64,
- MLX4_MFUNC_MAX = 80,
+ MLX4_MFUNC_MAX = 128,
MLX4_MAX_EQ_NUM = 1024,
MLX4_MFUNC_EQ_NUM = 4,
MLX4_MFUNC_MAX_EQES = 8,
#define VM_FAULT_WRITE 0x0008 /* Special case for get_user_pages */
#define VM_FAULT_HWPOISON 0x0010 /* Hit poisoned small page */
#define VM_FAULT_HWPOISON_LARGE 0x0020 /* Hit poisoned large page. Index encoded in upper bits */
+#define VM_FAULT_SIGSEGV 0x0040
#define VM_FAULT_NOPAGE 0x0100 /* ->fault installed the pte, not return page */
#define VM_FAULT_LOCKED 0x0200 /* ->fault locked the returned page */
#define VM_FAULT_HWPOISON_LARGE_MASK 0xf000 /* encodes hpage index for large hwpoison */
-#define VM_FAULT_ERROR (VM_FAULT_OOM | VM_FAULT_SIGBUS | VM_FAULT_HWPOISON | \
- VM_FAULT_FALLBACK | VM_FAULT_HWPOISON_LARGE)
+#define VM_FAULT_ERROR (VM_FAULT_OOM | VM_FAULT_SIGBUS | VM_FAULT_SIGSEGV | \
+ VM_FAULT_HWPOISON | VM_FAULT_HWPOISON_LARGE | \
+ VM_FAULT_FALLBACK)
/* Encode hstate index for a hwpoisoned large page */
#define VM_FAULT_SET_HINDEX(x) ((x) << 12)
* An MCS like lock especially tailored for optimistic spinning for sleeping
* lock implementations (mutex, rwsem, etc).
*/
-
-#define OSQ_UNLOCKED_VAL (0)
+struct optimistic_spin_node {
+ struct optimistic_spin_node *next, *prev;
+ int locked; /* 1 if lock acquired */
+ int cpu; /* encoded CPU # + 1 value */
+};
struct optimistic_spin_queue {
/*
atomic_t tail;
};
+#define OSQ_UNLOCKED_VAL (0)
+
/* Init macro and function. */
#define OSQ_LOCK_UNLOCKED { ATOMIC_INIT(OSQ_UNLOCKED_VAL) }
atomic_set(&lock->tail, OSQ_UNLOCKED_VAL);
}
+extern bool osq_lock(struct optimistic_spin_queue *lock);
+extern void osq_unlock(struct optimistic_spin_queue *lock);
+
#endif
#endif /* CONFIG_PERF_EVENTS */
};
-enum perf_event_context_type {
- task_context,
- cpu_context,
-};
-
/**
* struct perf_event_context - event context structure
*
*/
struct perf_event_context {
struct pmu *pmu;
- enum perf_event_context_type type;
/*
* Protect the states of the events in the list,
* nr_active, and the list:
*/
struct mutex mutex;
+ struct list_head active_ctx_list;
struct list_head pinned_groups;
struct list_head flexible_groups;
struct list_head event_list;
int exclusive;
struct hrtimer hrtimer;
ktime_t hrtimer_interval;
- struct list_head rotation_list;
struct pmu *unique_pmu;
struct perf_cgroup *cgrp;
};
extern struct static_key perf_swevent_enabled[PERF_COUNT_SW_MAX];
+extern void ___perf_sw_event(u32, u64, struct pt_regs *, u64);
extern void __perf_sw_event(u32, u64, struct pt_regs *, u64);
#ifndef perf_arch_fetch_caller_regs
static __always_inline void
perf_sw_event(u32 event_id, u64 nr, struct pt_regs *regs, u64 addr)
{
- struct pt_regs hot_regs;
+ if (static_key_false(&perf_swevent_enabled[event_id]))
+ __perf_sw_event(event_id, nr, regs, addr);
+}
+
+DECLARE_PER_CPU(struct pt_regs, __perf_regs[4]);
+/*
+ * 'Special' version for the scheduler, it hard assumes no recursion,
+ * which is guaranteed by us not actually scheduling inside other swevents
+ * because those disable preemption.
+ */
+static __always_inline void
+perf_sw_event_sched(u32 event_id, u64 nr, u64 addr)
+{
if (static_key_false(&perf_swevent_enabled[event_id])) {
- if (!regs) {
- perf_fetch_caller_regs(&hot_regs);
- regs = &hot_regs;
- }
- __perf_sw_event(event_id, nr, regs, addr);
+ struct pt_regs *regs = this_cpu_ptr(&__perf_regs[0]);
+
+ perf_fetch_caller_regs(regs);
+ ___perf_sw_event(event_id, nr, regs, addr);
}
}
static inline void perf_event_task_sched_out(struct task_struct *prev,
struct task_struct *next)
{
- perf_sw_event(PERF_COUNT_SW_CONTEXT_SWITCHES, 1, NULL, 0);
+ perf_sw_event_sched(PERF_COUNT_SW_CONTEXT_SWITCHES, 1, 0);
if (static_key_false(&perf_sched_events.key))
__perf_event_task_sched_out(prev, next);
static inline void
perf_sw_event(u32 event_id, u64 nr, struct pt_regs *regs, u64 addr) { }
static inline void
+perf_sw_event_sched(u32 event_id, u64 nr, u64 addr) { }
+static inline void
perf_bp_event(struct perf_event *event, void *data) { }
static inline int perf_register_guest_info_callbacks
#define SSDR (0x10) /* SSP Data Write/Data Read Register */
#define SSTO (0x28) /* SSP Time Out Register */
+#define DDS_RATE (0x28) /* SSP DDS Clock Rate Register (Intel Quark) */
#define SSPSP (0x2C) /* SSP Programmable Serial Protocol */
#define SSTSA (0x30) /* SSP Tx Timeslot Active */
#define SSRSA (0x34) /* SSP Rx Timeslot Active */
struct path;
+/* Structure for communicating via ->get_dqblk() & ->set_dqblk() */
+struct qc_dqblk {
+ int d_fieldmask; /* mask of fields to change in ->set_dqblk() */
+ u64 d_spc_hardlimit; /* absolute limit on used space */
+ u64 d_spc_softlimit; /* preferred limit on used space */
+ u64 d_ino_hardlimit; /* maximum # allocated inodes */
+ u64 d_ino_softlimit; /* preferred inode limit */
+ u64 d_space; /* Space owned by the user */
+ u64 d_ino_count; /* # inodes owned by the user */
+ s64 d_ino_timer; /* zero if within inode limits */
+ /* if not, we refuse service */
+ s64 d_spc_timer; /* similar to above; for space */
+ int d_ino_warns; /* # warnings issued wrt num inodes */
+ int d_spc_warns; /* # warnings issued wrt used space */
+ u64 d_rt_spc_hardlimit; /* absolute limit on realtime space */
+ u64 d_rt_spc_softlimit; /* preferred limit on RT space */
+ u64 d_rt_space; /* realtime space owned */
+ s64 d_rt_spc_timer; /* similar to above; for RT space */
+ int d_rt_spc_warns; /* # warnings issued wrt RT space */
+};
+
+/* Field specifiers for ->set_dqblk() in struct qc_dqblk */
+#define QC_INO_SOFT (1<<0)
+#define QC_INO_HARD (1<<1)
+#define QC_SPC_SOFT (1<<2)
+#define QC_SPC_HARD (1<<3)
+#define QC_RT_SPC_SOFT (1<<4)
+#define QC_RT_SPC_HARD (1<<5)
+#define QC_LIMIT_MASK (QC_INO_SOFT | QC_INO_HARD | QC_SPC_SOFT | QC_SPC_HARD | \
+ QC_RT_SPC_SOFT | QC_RT_SPC_HARD)
+#define QC_SPC_TIMER (1<<6)
+#define QC_INO_TIMER (1<<7)
+#define QC_RT_SPC_TIMER (1<<8)
+#define QC_TIMER_MASK (QC_SPC_TIMER | QC_INO_TIMER | QC_RT_SPC_TIMER)
+#define QC_SPC_WARNS (1<<9)
+#define QC_INO_WARNS (1<<10)
+#define QC_RT_SPC_WARNS (1<<11)
+#define QC_WARNS_MASK (QC_SPC_WARNS | QC_INO_WARNS | QC_RT_SPC_WARNS)
+#define QC_SPACE (1<<12)
+#define QC_INO_COUNT (1<<13)
+#define QC_RT_SPACE (1<<14)
+#define QC_ACCT_MASK (QC_SPACE | QC_INO_COUNT | QC_RT_SPACE)
+
/* Operations handling requests from userspace */
struct quotactl_ops {
int (*quota_on)(struct super_block *, int, int, struct path *);
int (*quota_sync)(struct super_block *, int);
int (*get_info)(struct super_block *, int, struct if_dqinfo *);
int (*set_info)(struct super_block *, int, struct if_dqinfo *);
- int (*get_dqblk)(struct super_block *, struct kqid, struct fs_disk_quota *);
- int (*set_dqblk)(struct super_block *, struct kqid, struct fs_disk_quota *);
+ int (*get_dqblk)(struct super_block *, struct kqid, struct qc_dqblk *);
+ int (*set_dqblk)(struct super_block *, struct kqid, struct qc_dqblk *);
int (*get_xstate)(struct super_block *, struct fs_quota_stat *);
int (*set_xstate)(struct super_block *, unsigned int, int);
int (*get_xstatev)(struct super_block *, struct fs_quota_statv *);
int dquot_get_dqinfo(struct super_block *sb, int type, struct if_dqinfo *ii);
int dquot_set_dqinfo(struct super_block *sb, int type, struct if_dqinfo *ii);
int dquot_get_dqblk(struct super_block *sb, struct kqid id,
- struct fs_disk_quota *di);
+ struct qc_dqblk *di);
int dquot_set_dqblk(struct super_block *sb, struct kqid id,
- struct fs_disk_quota *di);
+ struct qc_dqblk *di);
int __dquot_transfer(struct inode *inode, struct dquot **transfer_to);
int dquot_transfer(struct inode *inode, struct iattr *iattr);
* @member: the name of the hlist_node within the struct.
*/
#define hlist_for_each_entry_continue_rcu(pos, member) \
- for (pos = hlist_entry_safe(rcu_dereference((pos)->member.next),\
- typeof(*(pos)), member); \
+ for (pos = hlist_entry_safe(rcu_dereference_raw(hlist_next_rcu( \
+ &(pos)->member)), typeof(*(pos)), member); \
pos; \
- pos = hlist_entry_safe(rcu_dereference((pos)->member.next),\
- typeof(*(pos)), member))
+ pos = hlist_entry_safe(rcu_dereference_raw(hlist_next_rcu( \
+ &(pos)->member)), typeof(*(pos)), member))
/**
* hlist_for_each_entry_continue_rcu_bh - iterate over a hlist continuing after current point
* @member: the name of the hlist_node within the struct.
*/
#define hlist_for_each_entry_continue_rcu_bh(pos, member) \
- for (pos = hlist_entry_safe(rcu_dereference_bh((pos)->member.next),\
- typeof(*(pos)), member); \
+ for (pos = hlist_entry_safe(rcu_dereference_bh(hlist_next_rcu( \
+ &(pos)->member)), typeof(*(pos)), member); \
pos; \
- pos = hlist_entry_safe(rcu_dereference_bh((pos)->member.next),\
- typeof(*(pos)), member))
+ pos = hlist_entry_safe(rcu_dereference_bh(hlist_next_rcu( \
+ &(pos)->member)), typeof(*(pos)), member))
/**
* hlist_for_each_entry_from_rcu - iterate over a hlist continuing from current point
extern struct srcu_struct tasks_rcu_exit_srcu;
#define rcu_note_voluntary_context_switch(t) \
do { \
+ rcu_all_qs(); \
if (ACCESS_ONCE((t)->rcu_tasks_holdout)) \
ACCESS_ONCE((t)->rcu_tasks_holdout) = false; \
} while (0)
#else /* #ifdef CONFIG_TASKS_RCU */
#define TASKS_RCU(x) do { } while (0)
-#define rcu_note_voluntary_context_switch(t) do { } while (0)
+#define rcu_note_voluntary_context_switch(t) rcu_all_qs()
#endif /* #else #ifdef CONFIG_TASKS_RCU */
/**
})
#define __rcu_dereference_check(p, c, space) \
({ \
- typeof(*p) *_________p1 = (typeof(*p) *__force)ACCESS_ONCE(p); \
+ /* Dependency order vs. p above. */ \
+ typeof(*p) *________p1 = (typeof(*p) *__force)lockless_dereference(p); \
rcu_lockdep_assert(c, "suspicious rcu_dereference_check() usage"); \
rcu_dereference_sparse(p, space); \
- smp_read_barrier_depends(); /* Dependency order vs. p above. */ \
- ((typeof(*p) __force __kernel *)(_________p1)); \
+ ((typeof(*p) __force __kernel *)(________p1)); \
})
#define __rcu_dereference_protected(p, c, space) \
({ \
})
#define __rcu_dereference_index_check(p, c) \
({ \
- typeof(p) _________p1 = ACCESS_ONCE(p); \
+ /* Dependency order vs. p above. */ \
+ typeof(p) _________p1 = lockless_dereference(p); \
rcu_lockdep_assert(c, \
"suspicious rcu_dereference_index_check() usage"); \
- smp_read_barrier_depends(); /* Dependency order vs. p above. */ \
(_________p1); \
})
}
/*
- * Return the number of grace periods.
+ * Return the number of grace periods started.
*/
-static inline long rcu_batches_completed(void)
+static inline unsigned long rcu_batches_started(void)
{
return 0;
}
/*
- * Return the number of bottom-half grace periods.
+ * Return the number of bottom-half grace periods started.
*/
-static inline long rcu_batches_completed_bh(void)
+static inline unsigned long rcu_batches_started_bh(void)
+{
+ return 0;
+}
+
+/*
+ * Return the number of sched grace periods started.
+ */
+static inline unsigned long rcu_batches_started_sched(void)
+{
+ return 0;
+}
+
+/*
+ * Return the number of grace periods completed.
+ */
+static inline unsigned long rcu_batches_completed(void)
+{
+ return 0;
+}
+
+/*
+ * Return the number of bottom-half grace periods completed.
+ */
+static inline unsigned long rcu_batches_completed_bh(void)
+{
+ return 0;
+}
+
+/*
+ * Return the number of sched grace periods completed.
+ */
+static inline unsigned long rcu_batches_completed_sched(void)
{
return 0;
}
return true;
}
-
#endif /* #else defined(CONFIG_DEBUG_LOCK_ALLOC) || defined(CONFIG_RCU_TRACE) */
+static inline void rcu_all_qs(void)
+{
+}
+
#endif /* __LINUX_RCUTINY_H */
extern unsigned long rcutorture_testseq;
extern unsigned long rcutorture_vernum;
-long rcu_batches_completed(void);
-long rcu_batches_completed_bh(void);
-long rcu_batches_completed_sched(void);
+unsigned long rcu_batches_started(void);
+unsigned long rcu_batches_started_bh(void);
+unsigned long rcu_batches_started_sched(void);
+unsigned long rcu_batches_completed(void);
+unsigned long rcu_batches_completed_bh(void);
+unsigned long rcu_batches_completed_sched(void);
void show_rcu_gp_kthreads(void);
void rcu_force_quiescent_state(void);
bool rcu_is_watching(void);
+void rcu_all_qs(void);
+
#endif /* __LINUX_RCUTREE_H */
*
* @reg: Offset of the register within the regmap bank
* @lsb: lsb of the register field.
- * @reg: msb of the register field.
+ * @msb: msb of the register field.
* @id_size: port size if it has some ports
* @id_offset: address offset for each ports
*/
* 2 : 2 phase 2 buck
*/
int num_buck;
+ int gpio_ren[DA9211_MAX_REGULATORS];
+ struct device_node *reg_node[DA9211_MAX_REGULATORS];
struct regulator_init_data *init_data[DA9211_MAX_REGULATORS];
};
#endif
struct regmap;
struct regulator_dev;
+struct regulator_config;
struct regulator_init_data;
struct regulator_enable_gpio;
* @supply_name: Identifying the regulator supply
* @of_match: Name used to identify regulator in DT.
* @regulators_node: Name of node containing regulator definitions in DT.
+ * @of_parse_cb: Optional callback called only if of_match is present.
+ * Will be called for each regulator parsed from DT, during
+ * init_data parsing.
+ * The regulator_config passed as argument to the callback will
+ * be a copy of config passed to regulator_register, valid only
+ * for this particular call. Callback may freely change the
+ * config but it cannot store it for later usage.
+ * Callback should return 0 on success or negative ERRNO
+ * indicating failure.
* @id: Numerical identifier for the regulator.
* @ops: Regulator operations table.
* @irq: Interrupt number for the regulator.
const char *supply_name;
const char *of_match;
const char *regulators_node;
+ int (*of_parse_cb)(struct device_node *,
+ const struct regulator_desc *,
+ struct regulator_config *);
int id;
bool continuous_voltage_range;
unsigned n_voltages;
void *driver_data; /* core does not touch this */
};
-int regulator_suspend_prepare(suspend_state_t state);
-int regulator_suspend_finish(void);
-
#ifdef CONFIG_REGULATOR
void regulator_has_full_constraints(void);
+int regulator_suspend_prepare(suspend_state_t state);
+int regulator_suspend_finish(void);
#else
static inline void regulator_has_full_constraints(void)
{
}
+static inline int regulator_suspend_prepare(suspend_state_t state)
+{
+ return 0;
+}
+static inline int regulator_suspend_finish(void)
+{
+ return 0;
+}
#endif
#endif
--- /dev/null
+/*
+ * Copyright (c) 2014 MediaTek Inc.
+ * Author: Flora Fu <flora.fu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __LINUX_REGULATOR_MT6397_H
+#define __LINUX_REGULATOR_MT6397_H
+
+enum {
+ MT6397_ID_VPCA15 = 0,
+ MT6397_ID_VPCA7,
+ MT6397_ID_VSRAMCA15,
+ MT6397_ID_VSRAMCA7,
+ MT6397_ID_VCORE,
+ MT6397_ID_VGPU,
+ MT6397_ID_VDRM,
+ MT6397_ID_VIO18 = 7,
+ MT6397_ID_VTCXO,
+ MT6397_ID_VA28,
+ MT6397_ID_VCAMA,
+ MT6397_ID_VIO28,
+ MT6397_ID_VUSB,
+ MT6397_ID_VMC,
+ MT6397_ID_VMCH,
+ MT6397_ID_VEMC3V3,
+ MT6397_ID_VGP1,
+ MT6397_ID_VGP2,
+ MT6397_ID_VGP3,
+ MT6397_ID_VGP4,
+ MT6397_ID_VGP5,
+ MT6397_ID_VGP6,
+ MT6397_ID_VIBR,
+ MT6397_ID_RG_MAX,
+};
+
+#define MT6397_MAX_REGULATOR MT6397_ID_RG_MAX
+#define MT6397_REGULATOR_ID97 0x97
+#define MT6397_REGULATOR_ID91 0x91
+
+#endif /* __LINUX_REGULATOR_MT6397_H */
#define PFUZE200_VGEN5 11
#define PFUZE200_VGEN6 12
+#define PFUZE3000_SW1A 0
+#define PFUZE3000_SW1B 1
+#define PFUZE3000_SW2 2
+#define PFUZE3000_SW3 3
+#define PFUZE3000_SWBST 4
+#define PFUZE3000_VSNVS 5
+#define PFUZE3000_VREFDDR 6
+#define PFUZE3000_VLDO1 7
+#define PFUZE3000_VLDO2 8
+#define PFUZE3000_VCCSD 9
+#define PFUZE3000_V33 10
+#define PFUZE3000_VLDO3 11
+#define PFUZE3000_VLDO4 12
+
struct regulator_init_data;
struct pfuze_regulator_platform_data {
extern int rtc_read_time(struct rtc_device *rtc, struct rtc_time *tm);
extern int rtc_set_time(struct rtc_device *rtc, struct rtc_time *tm);
extern int rtc_set_mmss(struct rtc_device *rtc, unsigned long secs);
-extern int rtc_set_ntp_time(struct timespec now);
+extern int rtc_set_ntp_time(struct timespec64 now);
int __rtc_read_alarm(struct rtc_device *rtc, struct rtc_wkalrm *alarm);
extern int rtc_read_alarm(struct rtc_device *rtc,
struct rtc_wkalrm *alrm);
static inline void kick_all_cpus_sync(void) { }
static inline void wake_up_all_idle_cpus(void) { }
+#ifdef CONFIG_UP_LATE_INIT
+extern void __init up_late_init(void);
+static inline void smp_init(void) { up_late_init(); }
+#else
+static inline void smp_init(void) { }
+#endif
+
#endif /* !SMP */
/*
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
- *
* Written by:
* Dmitry Eremin-Solenikov <dmitry.baryshkov@siemens.com>
*/
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#ifndef _INCLUDE_LINUX_SPI_L4F00242T03_H_
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#ifndef _INCLUDE_LINUX_SPI_LMS283GF05_H_
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
*/
#ifndef __LINUX_SPI_MXS_SPI_H__
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
#ifndef __linux_pxa2xx_spi_h
#define __linux_pxa2xx_spi_h
#if defined(CONFIG_ARCH_PXA) || defined(CONFIG_ARCH_MMP)
#include <linux/clk.h>
-#include <mach/dma.h>
extern void pxa2xx_set_spi_info(unsigned id, struct pxa2xx_spi_master *info);
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
*/
#ifndef __LINUX_SPI_RENESAS_SPI_H__
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
*/
#ifndef SH_HSPI_H
#define SH_HSPI_H
u16 num_chipselect;
unsigned int dma_tx_id;
unsigned int dma_rx_id;
+ u32 dtdl;
+ u32 syncdl;
};
#endif /* __SPI_SH_MSIOF_H__ */
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
#ifndef __LINUX_SPI_H
* @pump_messages: work struct for scheduling work to the message pump
* @queue_lock: spinlock to syncronise access to message queue
* @queue: message queue
+ * @idling: the device is entering idle state
* @cur_msg: the currently in-flight message
* @cur_msg_prepared: spi_prepare_message was called for the currently
* in-flight message
spinlock_t queue_lock;
struct list_head queue;
struct spi_message *cur_msg;
+ bool idling;
bool busy;
bool running;
bool rt;
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
struct tle62x0_pdata {
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
*/
#ifndef _LINUX_SPI_TSC2005_H
#define RCU_BATCH_INIT(name) { NULL, &(name.head) }
struct srcu_struct {
- unsigned completed;
+ unsigned long completed;
struct srcu_struct_array __percpu *per_cpu_ref;
spinlock_t queue_lock; /* protect ->batch_queue, ->running */
bool running;
* define and init a srcu struct at build time.
* dont't call init_srcu_struct() nor cleanup_srcu_struct() on it.
*/
-#define DEFINE_SRCU(name) \
+#define __DEFINE_SRCU(name, is_static) \
static DEFINE_PER_CPU(struct srcu_struct_array, name##_srcu_array);\
- struct srcu_struct name = __SRCU_STRUCT_INIT(name);
-
-#define DEFINE_STATIC_SRCU(name) \
- static DEFINE_PER_CPU(struct srcu_struct_array, name##_srcu_array);\
- static struct srcu_struct name = __SRCU_STRUCT_INIT(name);
+ is_static struct srcu_struct name = __SRCU_STRUCT_INIT(name)
+#define DEFINE_SRCU(name) __DEFINE_SRCU(name, /* not static */)
+#define DEFINE_STATIC_SRCU(name) __DEFINE_SRCU(name, static)
/**
* call_srcu() - Queue a callback for invocation after an SRCU grace period
void __srcu_read_unlock(struct srcu_struct *sp, int idx) __releases(sp);
void synchronize_srcu(struct srcu_struct *sp);
void synchronize_srcu_expedited(struct srcu_struct *sp);
-long srcu_batches_completed(struct srcu_struct *sp);
+unsigned long srcu_batches_completed(struct srcu_struct *sp);
void srcu_barrier(struct srcu_struct *sp);
#ifdef CONFIG_DEBUG_LOCK_ALLOC
extern int __getnstimeofday64(struct timespec64 *tv);
extern void getnstimeofday64(struct timespec64 *tv);
+extern void getboottime64(struct timespec64 *ts);
#if BITS_PER_LONG == 64
/**
{
return get_monotonic_coarse64();
}
+
+static inline void getboottime(struct timespec *ts)
+{
+ return getboottime64(ts);
+}
#else
/**
* Deprecated. Use do_settimeofday64().
{
return timespec64_to_timespec(get_monotonic_coarse64());
}
-#endif
-extern void getboottime(struct timespec *ts);
+static inline void getboottime(struct timespec *ts)
+{
+ struct timespec64 ts64;
+
+ getboottime64(&ts64);
+ *ts = timespec64_to_timespec(ts64);
+}
+#endif
#define do_posix_clock_monotonic_gettime(ts) ktime_get_ts(ts)
#define ktime_get_real_ts64(ts) getnstimeofday64(ts)
*ts = ktime_to_timespec(ktime_get_boottime());
}
+static inline void get_monotonic_boottime64(struct timespec64 *ts)
+{
+ *ts = ktime_to_timespec64(ktime_get_boottime());
+}
+
static inline void timekeeping_clocktai(struct timespec *ts)
{
*ts = ktime_to_timespec(ktime_get_clocktai());
TP_PROTO(data_proto), \
TP_ARGS(data_args), \
TP_CONDITION(cond),,); \
- if (IS_ENABLED(CONFIG_LOCKDEP)) { \
+ if (IS_ENABLED(CONFIG_LOCKDEP) && (cond)) { \
rcu_read_lock_sched_notrace(); \
rcu_dereference_sched(__tracepoint_##name.funcs);\
rcu_read_unlock_sched_notrace(); \
*/
#define wait_event_cmd(wq, condition, cmd1, cmd2) \
do { \
- might_sleep(); \
if (condition) \
break; \
__wait_event_cmd(wq, condition, cmd1, cmd2); \
mode);
}
+/**
+ * wait_on_bit_timeout - wait for a bit to be cleared or a timeout elapses
+ * @word: the word being waited on, a kernel virtual address
+ * @bit: the bit of the word being waited on
+ * @mode: the task state to sleep in
+ * @timeout: timeout, in jiffies
+ *
+ * Use the standard hashed waitqueue table to wait for a bit
+ * to be cleared. This is similar to wait_on_bit(), except also takes a
+ * timeout parameter.
+ *
+ * Returned value will be zero if the bit was cleared before the
+ * @timeout elapsed, or non-zero if the @timeout elapsed or process
+ * received a signal and the mode permitted wakeup on that signal.
+ */
+static inline int
+wait_on_bit_timeout(void *word, int bit, unsigned mode, unsigned long timeout)
+{
+ might_sleep();
+ if (!test_bit(bit, word))
+ return 0;
+ return out_of_line_wait_on_bit_timeout(word, bit,
+ bit_wait_timeout,
+ mode, timeout);
+}
+
/**
* wait_on_bit_action - wait for a bit to be cleared
* @word: the word being waited on, a kernel virtual address
__be32 ports;
__be16 port16[2];
};
- u16 thoff;
- u16 n_proto;
- u8 ip_proto;
+ u16 thoff;
+ __be16 n_proto;
+ u8 ip_proto;
};
bool __skb_flow_dissect(const struct sk_buff *skb, struct flow_keys *flow,
return (arg->flags & IP_REPLY_ARG_NOSRCCHECK) ? FLOWI_FLAG_ANYSRC : 0;
}
-void ip_send_unicast_reply(struct net *net, struct sk_buff *skb,
+void ip_send_unicast_reply(struct sock *sk, struct sk_buff *skb,
const struct ip_options *sopt,
__be32 daddr, __be32 saddr,
const struct ip_reply_arg *arg,
return __ipv6_addr_diff(a1, a2, sizeof(struct in6_addr));
}
+u32 __ipv6_select_ident(u32 hashrnd, struct in6_addr *dst,
+ struct in6_addr *src);
+void ipv6_select_ident(struct frag_hdr *fhdr, struct rt6_info *rt);
void ipv6_proxy_select_ident(struct sk_buff *skb);
int ip6_dst_hoplimit(struct dst_entry *dst);
__be32 flowlabel, bool autolabel)
{
if (!flowlabel && (autolabel || net->ipv6.sysctl.auto_flowlabels)) {
- __be32 hash;
+ u32 hash;
hash = skb_get_hash(skb);
*/
hash ^= hash >> 12;
- flowlabel = hash & IPV6_FLOWLABEL_MASK;
+ flowlabel = (__force __be32)hash & IPV6_FLOWLABEL_MASK;
}
return flowlabel;
int nft_chain_validate_dependency(const struct nft_chain *chain,
enum nft_chain_type type);
+int nft_chain_validate_hooks(const struct nft_chain *chain,
+ unsigned int hook_flags);
struct nft_stats {
u64 bytes;
struct inet_peer_base *peers;
struct tcpm_hash_bucket *tcp_metrics_hash;
unsigned int tcp_metrics_hash_log;
+ struct sock * __percpu *tcp_sk;
struct netns_frags frags;
#ifdef CONFIG_NETFILTER
struct xt_table *iptable_filter;
struct netdev_queue *dev_queue;
struct gnet_stats_rate_est64 rate_est;
+ struct gnet_stats_basic_cpu __percpu *cpu_bstats;
+ struct gnet_stats_queue __percpu *cpu_qstats;
+
struct Qdisc *next_sched;
struct sk_buff *gso_skb;
/*
*/
unsigned long state;
struct sk_buff_head q;
- union {
- struct gnet_stats_basic_packed bstats;
- struct gnet_stats_basic_cpu __percpu *cpu_bstats;
- } __packed;
+ struct gnet_stats_basic_packed bstats;
unsigned int __state;
- union {
- struct gnet_stats_queue qstats;
- struct gnet_stats_queue __percpu *cpu_qstats;
- } __packed;
+ struct gnet_stats_queue qstats;
struct rcu_head rcu_head;
int padded;
atomic_t refcnt;
void tcp_get_allowed_congestion_control(char *buf, size_t len);
int tcp_set_allowed_congestion_control(char *allowed);
int tcp_set_congestion_control(struct sock *sk, const char *name);
-void tcp_slow_start(struct tcp_sock *tp, u32 acked);
-void tcp_cong_avoid_ai(struct tcp_sock *tp, u32 w);
+u32 tcp_slow_start(struct tcp_sock *tp, u32 acked);
+void tcp_cong_avoid_ai(struct tcp_sock *tp, u32 w, u32 acked);
u32 tcp_reno_ssthresh(struct sock *sk);
void tcp_reno_cong_avoid(struct sock *sk, u32 ack, u32 acked);
static inline int ib_copy_to_udata(struct ib_udata *udata, void *src, size_t len)
{
- size_t copy_sz;
-
- copy_sz = min_t(size_t, len, udata->outlen);
- return copy_to_user(udata->outbuf, src, copy_sz) ? -EFAULT : 0;
+ return copy_to_user(udata->outbuf, src, len) ? -EFAULT : 0;
}
/**
ak4113_write_t *write;
ak4113_read_t *read;
void *private_data;
- unsigned int init:1;
+ atomic_t wq_processing;
spinlock_t lock;
unsigned char regmap[AK4113_WRITABLE_REGS];
struct snd_kcontrol *kctls[AK4113_CONTROLS];
ak4114_write_t * write;
ak4114_read_t * read;
void * private_data;
- unsigned int init: 1;
+ atomic_t wq_processing;
spinlock_t lock;
unsigned char regmap[6];
unsigned char txcsb[5];
unsigned int mask, unsigned int value);
#ifdef CONFIG_SND_SOC_AC97_BUS
+struct snd_ac97 *snd_soc_alloc_ac97_codec(struct snd_soc_codec *codec);
struct snd_ac97 *snd_soc_new_ac97_codec(struct snd_soc_codec *codec);
void snd_soc_free_ac97_codec(struct snd_ac97 *ac97);
{ TLB_LOCAL_SHOOTDOWN, "local shootdown" }, \
{ TLB_LOCAL_MM_SHOOTDOWN, "local mm shootdown" }
-TRACE_EVENT(tlb_flush,
+TRACE_EVENT_CONDITION(tlb_flush,
TP_PROTO(int reason, unsigned long pages),
TP_ARGS(reason, pages),
+ TP_CONDITION(cpu_online(smp_processor_id())),
+
TP_STRUCT__entry(
__field( int, reason)
__field(unsigned long, pages)
struct ftrace_event_call *event_call = __data; \
struct ftrace_data_offsets_##call __maybe_unused __data_offsets;\
struct ftrace_raw_##call *entry; \
- struct pt_regs __regs; \
+ struct pt_regs *__regs; \
u64 __addr = 0, __count = 1; \
struct task_struct *__task = NULL; \
struct hlist_head *head; \
sizeof(u64)); \
__entry_size -= sizeof(u32); \
\
- perf_fetch_caller_regs(&__regs); \
entry = perf_trace_buf_prepare(__entry_size, \
event_call->event.type, &__regs, &rctx); \
if (!entry) \
return; \
\
+ perf_fetch_caller_regs(__regs); \
+ \
tstruct \
\
{ assign; } \
\
perf_trace_buf_submit(entry, __entry_size, rctx, __addr, \
- __count, &__regs, head, __task); \
+ __count, __regs, head, __task); \
}
/*
};
enum {
- IB_USER_VERBS_EX_CMD_QUERY_DEVICE = IB_USER_VERBS_CMD_QUERY_DEVICE,
IB_USER_VERBS_EX_CMD_CREATE_FLOW = IB_USER_VERBS_CMD_THRESHOLD,
IB_USER_VERBS_EX_CMD_DESTROY_FLOW,
};
__u8 reserved[4];
};
-enum {
- IB_USER_VERBS_EX_QUERY_DEVICE_ODP = 1ULL << 0,
-};
-
-struct ib_uverbs_ex_query_device {
- __u32 comp_mask;
- __u32 reserved;
-};
-
-struct ib_uverbs_odp_caps {
- __u64 general_caps;
- struct {
- __u32 rc_odp_caps;
- __u32 uc_odp_caps;
- __u32 ud_odp_caps;
- } per_transport_caps;
- __u32 reserved;
-};
-
-struct ib_uverbs_ex_query_device_resp {
- struct ib_uverbs_query_device_resp base;
- __u32 comp_mask;
- __u32 reserved;
- struct ib_uverbs_odp_caps odp_caps;
-};
-
struct ib_uverbs_query_port {
__u64 response;
__u8 port_num;
config TREE_RCU
bool "Tree-based hierarchical RCU"
depends on !PREEMPT && SMP
- select IRQ_WORK
help
This option selects the RCU implementation that is
designed for very large SMP system with hundreds or
config PREEMPT_RCU
bool "Preemptible tree-based hierarchical RCU"
depends on PREEMPT
- select IRQ_WORK
help
This option selects the RCU implementation that is
designed for very large SMP systems with hundreds or
endchoice
+config SRCU
+ bool
+ help
+ This option selects the sleepable version of RCU. This version
+ permits arbitrary sleeping or blocking within RCU read-side critical
+ sections.
+
config TASKS_RCU
bool "Task_based RCU implementation using voluntary context switch"
default n
+ select SRCU
help
This option enables a task-based RCU implementation that uses
only voluntary context switch (not preemption!), idle, and
config RCU_KTHREAD_PRIO
int "Real-time priority to use for RCU worker threads"
- range 1 99
- depends on RCU_BOOST
- default 1
+ range 1 99 if RCU_BOOST
+ range 0 99 if !RCU_BOOST
+ default 1 if RCU_BOOST
+ default 0 if !RCU_BOOST
help
This option specifies the SCHED_FIFO priority value that will be
assigned to the rcuc/n and rcub/n threads and is also the value
depends on HAVE_PERF_EVENTS
select ANON_INODES
select IRQ_WORK
+ select SRCU
help
Enable kernel support for various performance events provided
by software and hardware.
#include <asm/sections.h>
#include <asm/cacheflush.h>
-#ifdef CONFIG_X86_LOCAL_APIC
-#include <asm/smp.h>
-#endif
-
static int kernel_init(void *);
extern void init_IRQ(void);
#ifndef CONFIG_SMP
static const unsigned int setup_max_cpus = NR_CPUS;
-#ifdef CONFIG_X86_LOCAL_APIC
-static void __init smp_init(void)
-{
- APIC_init_uniprocessor();
-}
-#else
-#define smp_init() do { } while (0)
-#endif
-
static inline void setup_nr_cpu_ids(void) { }
static inline void smp_prepare_cpus(unsigned int maxcpus) { }
#endif
def_bool y
depends on SMP && RWSEM_XCHGADD_ALGORITHM && ARCH_SUPPORTS_ATOMIC_RMW
+config LOCK_SPIN_ON_OWNER
+ def_bool y
+ depends on MUTEX_SPIN_ON_OWNER || RWSEM_SPIN_ON_OWNER
+
config ARCH_USE_QUEUE_RWLOCK
bool
static struct {
struct task_struct *active_writer;
- struct mutex lock; /* Synchronizes accesses to refcount, */
+ /* wait queue to wake up the active_writer */
+ wait_queue_head_t wq;
+ /* verifies that no writer will get active while readers are active */
+ struct mutex lock;
/*
* Also blocks the new readers during
* an ongoing cpu hotplug operation.
*/
- int refcount;
- /* And allows lockless put_online_cpus(). */
- atomic_t puts_pending;
+ atomic_t refcount;
#ifdef CONFIG_DEBUG_LOCK_ALLOC
struct lockdep_map dep_map;
#endif
} cpu_hotplug = {
.active_writer = NULL,
+ .wq = __WAIT_QUEUE_HEAD_INITIALIZER(cpu_hotplug.wq),
.lock = __MUTEX_INITIALIZER(cpu_hotplug.lock),
- .refcount = 0,
#ifdef CONFIG_DEBUG_LOCK_ALLOC
.dep_map = {.name = "cpu_hotplug.lock" },
#endif
#define cpuhp_lock_acquire() lock_map_acquire(&cpu_hotplug.dep_map)
#define cpuhp_lock_release() lock_map_release(&cpu_hotplug.dep_map)
-static void apply_puts_pending(int max)
-{
- int delta;
-
- if (atomic_read(&cpu_hotplug.puts_pending) >= max) {
- delta = atomic_xchg(&cpu_hotplug.puts_pending, 0);
- cpu_hotplug.refcount -= delta;
- }
-}
void get_online_cpus(void)
{
return;
cpuhp_lock_acquire_read();
mutex_lock(&cpu_hotplug.lock);
- apply_puts_pending(65536);
- cpu_hotplug.refcount++;
+ atomic_inc(&cpu_hotplug.refcount);
mutex_unlock(&cpu_hotplug.lock);
}
EXPORT_SYMBOL_GPL(get_online_cpus);
if (!mutex_trylock(&cpu_hotplug.lock))
return false;
cpuhp_lock_acquire_tryread();
- apply_puts_pending(65536);
- cpu_hotplug.refcount++;
+ atomic_inc(&cpu_hotplug.refcount);
mutex_unlock(&cpu_hotplug.lock);
return true;
}
void put_online_cpus(void)
{
+ int refcount;
+
if (cpu_hotplug.active_writer == current)
return;
- if (!mutex_trylock(&cpu_hotplug.lock)) {
- atomic_inc(&cpu_hotplug.puts_pending);
- cpuhp_lock_release();
- return;
- }
- if (WARN_ON(!cpu_hotplug.refcount))
- cpu_hotplug.refcount++; /* try to fix things up */
+ refcount = atomic_dec_return(&cpu_hotplug.refcount);
+ if (WARN_ON(refcount < 0)) /* try to fix things up */
+ atomic_inc(&cpu_hotplug.refcount);
+
+ if (refcount <= 0 && waitqueue_active(&cpu_hotplug.wq))
+ wake_up(&cpu_hotplug.wq);
- if (!--cpu_hotplug.refcount && unlikely(cpu_hotplug.active_writer))
- wake_up_process(cpu_hotplug.active_writer);
- mutex_unlock(&cpu_hotplug.lock);
cpuhp_lock_release();
}
*/
void cpu_hotplug_begin(void)
{
- cpu_hotplug.active_writer = current;
+ DEFINE_WAIT(wait);
+ cpu_hotplug.active_writer = current;
cpuhp_lock_acquire();
+
for (;;) {
mutex_lock(&cpu_hotplug.lock);
- apply_puts_pending(1);
- if (likely(!cpu_hotplug.refcount))
- break;
- __set_current_state(TASK_UNINTERRUPTIBLE);
+ prepare_to_wait(&cpu_hotplug.wq, &wait, TASK_UNINTERRUPTIBLE);
+ if (likely(!atomic_read(&cpu_hotplug.refcount)))
+ break;
mutex_unlock(&cpu_hotplug.lock);
schedule();
}
+ finish_wait(&cpu_hotplug.wq, &wait);
}
void cpu_hotplug_done(void)
pmu->pmu_enable(pmu);
}
-static DEFINE_PER_CPU(struct list_head, rotation_list);
+static DEFINE_PER_CPU(struct list_head, active_ctx_list);
/*
- * perf_pmu_rotate_start() and perf_rotate_context() are fully serialized
- * because they're strictly cpu affine and rotate_start is called with IRQs
- * disabled, while rotate_context is called from IRQ context.
+ * perf_event_ctx_activate(), perf_event_ctx_deactivate(), and
+ * perf_event_task_tick() are fully serialized because they're strictly cpu
+ * affine and perf_event_ctx{activate,deactivate} are called with IRQs
+ * disabled, while perf_event_task_tick is called from IRQ context.
*/
-static void perf_pmu_rotate_start(struct pmu *pmu)
+static void perf_event_ctx_activate(struct perf_event_context *ctx)
{
- struct perf_cpu_context *cpuctx = this_cpu_ptr(pmu->pmu_cpu_context);
- struct list_head *head = this_cpu_ptr(&rotation_list);
+ struct list_head *head = this_cpu_ptr(&active_ctx_list);
WARN_ON(!irqs_disabled());
- if (list_empty(&cpuctx->rotation_list))
- list_add(&cpuctx->rotation_list, head);
+ WARN_ON(!list_empty(&ctx->active_ctx_list));
+
+ list_add(&ctx->active_ctx_list, head);
+}
+
+static void perf_event_ctx_deactivate(struct perf_event_context *ctx)
+{
+ WARN_ON(!irqs_disabled());
+
+ WARN_ON(list_empty(&ctx->active_ctx_list));
+
+ list_del_init(&ctx->active_ctx_list);
}
static void get_ctx(struct perf_event_context *ctx)
}
}
+/*
+ * Because of perf_event::ctx migration in sys_perf_event_open::move_group and
+ * perf_pmu_migrate_context() we need some magic.
+ *
+ * Those places that change perf_event::ctx will hold both
+ * perf_event_ctx::mutex of the 'old' and 'new' ctx value.
+ *
+ * Lock ordering is by mutex address. There is one other site where
+ * perf_event_context::mutex nests and that is put_event(). But remember that
+ * that is a parent<->child context relation, and migration does not affect
+ * children, therefore these two orderings should not interact.
+ *
+ * The change in perf_event::ctx does not affect children (as claimed above)
+ * because the sys_perf_event_open() case will install a new event and break
+ * the ctx parent<->child relation, and perf_pmu_migrate_context() is only
+ * concerned with cpuctx and that doesn't have children.
+ *
+ * The places that change perf_event::ctx will issue:
+ *
+ * perf_remove_from_context();
+ * synchronize_rcu();
+ * perf_install_in_context();
+ *
+ * to affect the change. The remove_from_context() + synchronize_rcu() should
+ * quiesce the event, after which we can install it in the new location. This
+ * means that only external vectors (perf_fops, prctl) can perturb the event
+ * while in transit. Therefore all such accessors should also acquire
+ * perf_event_context::mutex to serialize against this.
+ *
+ * However; because event->ctx can change while we're waiting to acquire
+ * ctx->mutex we must be careful and use the below perf_event_ctx_lock()
+ * function.
+ *
+ * Lock order:
+ * task_struct::perf_event_mutex
+ * perf_event_context::mutex
+ * perf_event_context::lock
+ * perf_event::child_mutex;
+ * perf_event::mmap_mutex
+ * mmap_sem
+ */
+static struct perf_event_context *
+perf_event_ctx_lock_nested(struct perf_event *event, int nesting)
+{
+ struct perf_event_context *ctx;
+
+again:
+ rcu_read_lock();
+ ctx = ACCESS_ONCE(event->ctx);
+ if (!atomic_inc_not_zero(&ctx->refcount)) {
+ rcu_read_unlock();
+ goto again;
+ }
+ rcu_read_unlock();
+
+ mutex_lock_nested(&ctx->mutex, nesting);
+ if (event->ctx != ctx) {
+ mutex_unlock(&ctx->mutex);
+ put_ctx(ctx);
+ goto again;
+ }
+
+ return ctx;
+}
+
+static inline struct perf_event_context *
+perf_event_ctx_lock(struct perf_event *event)
+{
+ return perf_event_ctx_lock_nested(event, 0);
+}
+
+static void perf_event_ctx_unlock(struct perf_event *event,
+ struct perf_event_context *ctx)
+{
+ mutex_unlock(&ctx->mutex);
+ put_ctx(ctx);
+}
+
/*
* This must be done under the ctx->lock, such as to serialize against
* context_equiv(), therefore we cannot call put_ctx() since that might end up
ctx->nr_branch_stack++;
list_add_rcu(&event->event_entry, &ctx->event_list);
- if (!ctx->nr_events)
- perf_pmu_rotate_start(ctx->pmu);
ctx->nr_events++;
if (event->attr.inherit_stat)
ctx->nr_stat++;
if (group_leader == event)
return;
+ WARN_ON_ONCE(group_leader->ctx != event->ctx);
+
if (group_leader->group_flags & PERF_GROUP_SOFTWARE &&
!is_software_event(event))
group_leader->group_flags &= ~PERF_GROUP_SOFTWARE;
list_del_event(struct perf_event *event, struct perf_event_context *ctx)
{
struct perf_cpu_context *cpuctx;
+
+ WARN_ON_ONCE(event->ctx != ctx);
+ lockdep_assert_held(&ctx->lock);
+
/*
* We can have double detach due to exit/hot-unplug + close.
*/
/* Inherit group flags from the previous leader */
sibling->group_flags = event->group_flags;
+
+ WARN_ON_ONCE(sibling->ctx != event->ctx);
}
out:
{
u64 tstamp = perf_event_time(event);
u64 delta;
+
+ WARN_ON_ONCE(event->ctx != ctx);
+ lockdep_assert_held(&ctx->lock);
+
/*
* An event which could not be activated because of
* filter mismatch still needs to have its timings
if (!is_software_event(event))
cpuctx->active_oncpu--;
- ctx->nr_active--;
+ if (!--ctx->nr_active)
+ perf_event_ctx_deactivate(ctx);
if (event->attr.freq && event->attr.sample_freq)
ctx->nr_freq--;
if (event->attr.exclusive || !cpuctx->active_oncpu)
* is the current context on this CPU and preemption is disabled,
* hence we can't get into perf_event_task_sched_out for this context.
*/
-void perf_event_disable(struct perf_event *event)
+static void _perf_event_disable(struct perf_event *event)
{
struct perf_event_context *ctx = event->ctx;
struct task_struct *task = ctx->task;
}
raw_spin_unlock_irq(&ctx->lock);
}
+
+/*
+ * Strictly speaking kernel users cannot create groups and therefore this
+ * interface does not need the perf_event_ctx_lock() magic.
+ */
+void perf_event_disable(struct perf_event *event)
+{
+ struct perf_event_context *ctx;
+
+ ctx = perf_event_ctx_lock(event);
+ _perf_event_disable(event);
+ perf_event_ctx_unlock(event, ctx);
+}
EXPORT_SYMBOL_GPL(perf_event_disable);
static void perf_set_shadow_time(struct perf_event *event,
if (!is_software_event(event))
cpuctx->active_oncpu++;
- ctx->nr_active++;
+ if (!ctx->nr_active++)
+ perf_event_ctx_activate(ctx);
if (event->attr.freq && event->attr.sample_freq)
ctx->nr_freq++;
* perf_event_for_each_child or perf_event_for_each as described
* for perf_event_disable.
*/
-void perf_event_enable(struct perf_event *event)
+static void _perf_event_enable(struct perf_event *event)
{
struct perf_event_context *ctx = event->ctx;
struct task_struct *task = ctx->task;
out:
raw_spin_unlock_irq(&ctx->lock);
}
+
+/*
+ * See perf_event_disable();
+ */
+void perf_event_enable(struct perf_event *event)
+{
+ struct perf_event_context *ctx;
+
+ ctx = perf_event_ctx_lock(event);
+ _perf_event_enable(event);
+ perf_event_ctx_unlock(event, ctx);
+}
EXPORT_SYMBOL_GPL(perf_event_enable);
-int perf_event_refresh(struct perf_event *event, int refresh)
+static int _perf_event_refresh(struct perf_event *event, int refresh)
{
/*
* not supported on inherited events
return -EINVAL;
atomic_add(refresh, &event->event_limit);
- perf_event_enable(event);
+ _perf_event_enable(event);
return 0;
}
+
+/*
+ * See perf_event_disable()
+ */
+int perf_event_refresh(struct perf_event *event, int refresh)
+{
+ struct perf_event_context *ctx;
+ int ret;
+
+ ctx = perf_event_ctx_lock(event);
+ ret = _perf_event_refresh(event, refresh);
+ perf_event_ctx_unlock(event, ctx);
+
+ return ret;
+}
EXPORT_SYMBOL_GPL(perf_event_refresh);
static void ctx_sched_out(struct perf_event_context *ctx,
perf_pmu_enable(ctx->pmu);
perf_ctx_unlock(cpuctx, ctx);
-
- /*
- * Since these rotations are per-cpu, we need to ensure the
- * cpu-context we got scheduled on is actually rotating.
- */
- perf_pmu_rotate_start(ctx->pmu);
}
/*
list_rotate_left(&ctx->flexible_groups);
}
-/*
- * perf_pmu_rotate_start() and perf_rotate_context() are fully serialized
- * because they're strictly cpu affine and rotate_start is called with IRQs
- * disabled, while rotate_context is called from IRQ context.
- */
static int perf_rotate_context(struct perf_cpu_context *cpuctx)
{
struct perf_event_context *ctx = NULL;
- int rotate = 0, remove = 1;
+ int rotate = 0;
if (cpuctx->ctx.nr_events) {
- remove = 0;
if (cpuctx->ctx.nr_events != cpuctx->ctx.nr_active)
rotate = 1;
}
ctx = cpuctx->task_ctx;
if (ctx && ctx->nr_events) {
- remove = 0;
if (ctx->nr_events != ctx->nr_active)
rotate = 1;
}
perf_pmu_enable(cpuctx->ctx.pmu);
perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
done:
- if (remove)
- list_del_init(&cpuctx->rotation_list);
return rotate;
}
void perf_event_task_tick(void)
{
- struct list_head *head = this_cpu_ptr(&rotation_list);
- struct perf_cpu_context *cpuctx, *tmp;
- struct perf_event_context *ctx;
+ struct list_head *head = this_cpu_ptr(&active_ctx_list);
+ struct perf_event_context *ctx, *tmp;
int throttled;
WARN_ON(!irqs_disabled());
__this_cpu_inc(perf_throttled_seq);
throttled = __this_cpu_xchg(perf_throttled_count, 0);
- list_for_each_entry_safe(cpuctx, tmp, head, rotation_list) {
- ctx = &cpuctx->ctx;
+ list_for_each_entry_safe(ctx, tmp, head, active_ctx_list)
perf_adjust_freq_unthr_context(ctx, throttled);
-
- ctx = cpuctx->task_ctx;
- if (ctx)
- perf_adjust_freq_unthr_context(ctx, throttled);
- }
}
static int event_enable_on_exec(struct perf_event *event,
{
raw_spin_lock_init(&ctx->lock);
mutex_init(&ctx->mutex);
+ INIT_LIST_HEAD(&ctx->active_ctx_list);
INIT_LIST_HEAD(&ctx->pinned_groups);
INIT_LIST_HEAD(&ctx->flexible_groups);
INIT_LIST_HEAD(&ctx->event_list);
rcu_read_unlock();
if (owner) {
- mutex_lock(&owner->perf_event_mutex);
+ /*
+ * If we're here through perf_event_exit_task() we're already
+ * holding ctx->mutex which would be an inversion wrt. the
+ * normal lock order.
+ *
+ * However we can safely take this lock because its the child
+ * ctx->mutex.
+ */
+ mutex_lock_nested(&owner->perf_event_mutex, SINGLE_DEPTH_NESTING);
+
/*
* We have to re-check the event->owner field, if it is cleared
* we raced with perf_event_exit_task(), acquiring the mutex
*/
static void put_event(struct perf_event *event)
{
- struct perf_event_context *ctx = event->ctx;
+ struct perf_event_context *ctx;
if (!atomic_long_dec_and_test(&event->refcount))
return;
if (!is_kernel_event(event))
perf_remove_from_owner(event);
- WARN_ON_ONCE(ctx->parent_ctx);
/*
* There are two ways this annotation is useful:
*
* the last filedesc died, so there is no possibility
* to trigger the AB-BA case.
*/
- mutex_lock_nested(&ctx->mutex, SINGLE_DEPTH_NESTING);
+ ctx = perf_event_ctx_lock_nested(event, SINGLE_DEPTH_NESTING);
+ WARN_ON_ONCE(ctx->parent_ctx);
perf_remove_from_context(event, true);
mutex_unlock(&ctx->mutex);
u64 read_format, char __user *buf)
{
struct perf_event *leader = event->group_leader, *sub;
- int n = 0, size = 0, ret = -EFAULT;
struct perf_event_context *ctx = leader->ctx;
- u64 values[5];
+ int n = 0, size = 0, ret;
u64 count, enabled, running;
+ u64 values[5];
+
+ lockdep_assert_held(&ctx->mutex);
- mutex_lock(&ctx->mutex);
count = perf_event_read_value(leader, &enabled, &running);
values[n++] = 1 + leader->nr_siblings;
size = n * sizeof(u64);
if (copy_to_user(buf, values, size))
- goto unlock;
+ return -EFAULT;
ret = size;
size = n * sizeof(u64);
if (copy_to_user(buf + ret, values, size)) {
- ret = -EFAULT;
- goto unlock;
+ return -EFAULT;
}
ret += size;
}
-unlock:
- mutex_unlock(&ctx->mutex);
return ret;
}
perf_read(struct file *file, char __user *buf, size_t count, loff_t *ppos)
{
struct perf_event *event = file->private_data;
+ struct perf_event_context *ctx;
+ int ret;
+
+ ctx = perf_event_ctx_lock(event);
+ ret = perf_read_hw(event, buf, count);
+ perf_event_ctx_unlock(event, ctx);
- return perf_read_hw(event, buf, count);
+ return ret;
}
static unsigned int perf_poll(struct file *file, poll_table *wait)
return events;
}
-static void perf_event_reset(struct perf_event *event)
+static void _perf_event_reset(struct perf_event *event)
{
(void)perf_event_read(event);
local64_set(&event->count, 0);
struct perf_event *child;
WARN_ON_ONCE(event->ctx->parent_ctx);
+
mutex_lock(&event->child_mutex);
func(event);
list_for_each_entry(child, &event->child_list, child_list)
struct perf_event_context *ctx = event->ctx;
struct perf_event *sibling;
- WARN_ON_ONCE(ctx->parent_ctx);
- mutex_lock(&ctx->mutex);
+ lockdep_assert_held(&ctx->mutex);
+
event = event->group_leader;
perf_event_for_each_child(event, func);
list_for_each_entry(sibling, &event->sibling_list, group_entry)
perf_event_for_each_child(sibling, func);
- mutex_unlock(&ctx->mutex);
}
static int perf_event_period(struct perf_event *event, u64 __user *arg)
struct perf_event *output_event);
static int perf_event_set_filter(struct perf_event *event, void __user *arg);
-static long perf_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
+static long _perf_ioctl(struct perf_event *event, unsigned int cmd, unsigned long arg)
{
- struct perf_event *event = file->private_data;
void (*func)(struct perf_event *);
u32 flags = arg;
switch (cmd) {
case PERF_EVENT_IOC_ENABLE:
- func = perf_event_enable;
+ func = _perf_event_enable;
break;
case PERF_EVENT_IOC_DISABLE:
- func = perf_event_disable;
+ func = _perf_event_disable;
break;
case PERF_EVENT_IOC_RESET:
- func = perf_event_reset;
+ func = _perf_event_reset;
break;
case PERF_EVENT_IOC_REFRESH:
- return perf_event_refresh(event, arg);
+ return _perf_event_refresh(event, arg);
case PERF_EVENT_IOC_PERIOD:
return perf_event_period(event, (u64 __user *)arg);
return 0;
}
+static long perf_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
+{
+ struct perf_event *event = file->private_data;
+ struct perf_event_context *ctx;
+ long ret;
+
+ ctx = perf_event_ctx_lock(event);
+ ret = _perf_ioctl(event, cmd, arg);
+ perf_event_ctx_unlock(event, ctx);
+
+ return ret;
+}
+
#ifdef CONFIG_COMPAT
static long perf_compat_ioctl(struct file *file, unsigned int cmd,
unsigned long arg)
int perf_event_task_enable(void)
{
+ struct perf_event_context *ctx;
struct perf_event *event;
mutex_lock(¤t->perf_event_mutex);
- list_for_each_entry(event, ¤t->perf_event_list, owner_entry)
- perf_event_for_each_child(event, perf_event_enable);
+ list_for_each_entry(event, ¤t->perf_event_list, owner_entry) {
+ ctx = perf_event_ctx_lock(event);
+ perf_event_for_each_child(event, _perf_event_enable);
+ perf_event_ctx_unlock(event, ctx);
+ }
mutex_unlock(¤t->perf_event_mutex);
return 0;
int perf_event_task_disable(void)
{
+ struct perf_event_context *ctx;
struct perf_event *event;
mutex_lock(¤t->perf_event_mutex);
- list_for_each_entry(event, ¤t->perf_event_list, owner_entry)
- perf_event_for_each_child(event, perf_event_disable);
+ list_for_each_entry(event, ¤t->perf_event_list, owner_entry) {
+ ctx = perf_event_ctx_lock(event);
+ perf_event_for_each_child(event, _perf_event_disable);
+ perf_event_ctx_unlock(event, ctx);
+ }
mutex_unlock(¤t->perf_event_mutex);
return 0;
rcu_read_unlock();
}
+DEFINE_PER_CPU(struct pt_regs, __perf_regs[4]);
+
int perf_swevent_get_recursion_context(void)
{
struct swevent_htable *swhash = this_cpu_ptr(&swevent_htable);
put_recursion_context(swhash->recursion, rctx);
}
-void __perf_sw_event(u32 event_id, u64 nr, struct pt_regs *regs, u64 addr)
+void ___perf_sw_event(u32 event_id, u64 nr, struct pt_regs *regs, u64 addr)
{
struct perf_sample_data data;
- int rctx;
- preempt_disable_notrace();
- rctx = perf_swevent_get_recursion_context();
- if (rctx < 0)
+ if (WARN_ON_ONCE(!regs))
return;
perf_sample_data_init(&data, addr, 0);
-
do_perf_sw_event(PERF_TYPE_SOFTWARE, event_id, nr, &data, regs);
+}
+
+void __perf_sw_event(u32 event_id, u64 nr, struct pt_regs *regs, u64 addr)
+{
+ int rctx;
+
+ preempt_disable_notrace();
+ rctx = perf_swevent_get_recursion_context();
+ if (unlikely(rctx < 0))
+ goto fail;
+
+ ___perf_sw_event(event_id, nr, regs, addr);
perf_swevent_put_recursion_context(rctx);
+fail:
preempt_enable_notrace();
}
__perf_event_init_context(&cpuctx->ctx);
lockdep_set_class(&cpuctx->ctx.mutex, &cpuctx_mutex);
lockdep_set_class(&cpuctx->ctx.lock, &cpuctx_lock);
- cpuctx->ctx.type = cpu_context;
cpuctx->ctx.pmu = pmu;
__perf_cpu_hrtimer_init(cpuctx, cpu);
- INIT_LIST_HEAD(&cpuctx->rotation_list);
cpuctx->unique_pmu = pmu;
}
}
EXPORT_SYMBOL_GPL(perf_pmu_unregister);
+static int perf_try_init_event(struct pmu *pmu, struct perf_event *event)
+{
+ int ret;
+
+ if (!try_module_get(pmu->module))
+ return -ENODEV;
+ event->pmu = pmu;
+ ret = pmu->event_init(event);
+ if (ret)
+ module_put(pmu->module);
+
+ return ret;
+}
+
struct pmu *perf_init_event(struct perf_event *event)
{
struct pmu *pmu = NULL;
pmu = idr_find(&pmu_idr, event->attr.type);
rcu_read_unlock();
if (pmu) {
- if (!try_module_get(pmu->module)) {
- pmu = ERR_PTR(-ENODEV);
- goto unlock;
- }
- event->pmu = pmu;
- ret = pmu->event_init(event);
+ ret = perf_try_init_event(pmu, event);
if (ret)
pmu = ERR_PTR(ret);
goto unlock;
}
list_for_each_entry_rcu(pmu, &pmus, entry) {
- if (!try_module_get(pmu->module)) {
- pmu = ERR_PTR(-ENODEV);
- goto unlock;
- }
- event->pmu = pmu;
- ret = pmu->event_init(event);
+ ret = perf_try_init_event(pmu, event);
if (!ret)
goto unlock;
return ret;
}
+static void mutex_lock_double(struct mutex *a, struct mutex *b)
+{
+ if (b < a)
+ swap(a, b);
+
+ mutex_lock(a);
+ mutex_lock_nested(b, SINGLE_DEPTH_NESTING);
+}
+
/**
* sys_perf_event_open - open a performance event, associate it to a task/cpu
*
struct perf_event *group_leader = NULL, *output_event = NULL;
struct perf_event *event, *sibling;
struct perf_event_attr attr;
- struct perf_event_context *ctx;
+ struct perf_event_context *ctx, *uninitialized_var(gctx);
struct file *event_file = NULL;
struct fd group = {NULL, 0};
struct task_struct *task = NULL;
* task or CPU context:
*/
if (move_group) {
- if (group_leader->ctx->type != ctx->type)
+ /*
+ * Make sure we're both on the same task, or both
+ * per-cpu events.
+ */
+ if (group_leader->ctx->task != ctx->task)
+ goto err_context;
+
+ /*
+ * Make sure we're both events for the same CPU;
+ * grouping events for different CPUs is broken; since
+ * you can never concurrently schedule them anyhow.
+ */
+ if (group_leader->cpu != event->cpu)
goto err_context;
} else {
if (group_leader->ctx != ctx)
}
if (move_group) {
- struct perf_event_context *gctx = group_leader->ctx;
-
- mutex_lock(&gctx->mutex);
- perf_remove_from_context(group_leader, false);
+ gctx = group_leader->ctx;
/*
- * Removing from the context ends up with disabled
- * event. What we want here is event in the initial
- * startup state, ready to be add into new context.
+ * See perf_event_ctx_lock() for comments on the details
+ * of swizzling perf_event::ctx.
*/
- perf_event__state_init(group_leader);
+ mutex_lock_double(&gctx->mutex, &ctx->mutex);
+
+ perf_remove_from_context(group_leader, false);
+
list_for_each_entry(sibling, &group_leader->sibling_list,
group_entry) {
perf_remove_from_context(sibling, false);
- perf_event__state_init(sibling);
put_ctx(gctx);
}
- mutex_unlock(&gctx->mutex);
- put_ctx(gctx);
+ } else {
+ mutex_lock(&ctx->mutex);
}
WARN_ON_ONCE(ctx->parent_ctx);
- mutex_lock(&ctx->mutex);
if (move_group) {
+ /*
+ * Wait for everybody to stop referencing the events through
+ * the old lists, before installing it on new lists.
+ */
synchronize_rcu();
- perf_install_in_context(ctx, group_leader, group_leader->cpu);
- get_ctx(ctx);
+
+ /*
+ * Install the group siblings before the group leader.
+ *
+ * Because a group leader will try and install the entire group
+ * (through the sibling list, which is still in-tact), we can
+ * end up with siblings installed in the wrong context.
+ *
+ * By installing siblings first we NO-OP because they're not
+ * reachable through the group lists.
+ */
list_for_each_entry(sibling, &group_leader->sibling_list,
group_entry) {
+ perf_event__state_init(sibling);
perf_install_in_context(ctx, sibling, sibling->cpu);
get_ctx(ctx);
}
+
+ /*
+ * Removing from the context ends up with disabled
+ * event. What we want here is event in the initial
+ * startup state, ready to be add into new context.
+ */
+ perf_event__state_init(group_leader);
+ perf_install_in_context(ctx, group_leader, group_leader->cpu);
+ get_ctx(ctx);
}
perf_install_in_context(ctx, event, event->cpu);
perf_unpin_context(ctx);
+
+ if (move_group) {
+ mutex_unlock(&gctx->mutex);
+ put_ctx(gctx);
+ }
mutex_unlock(&ctx->mutex);
put_online_cpus();
src_ctx = &per_cpu_ptr(pmu->pmu_cpu_context, src_cpu)->ctx;
dst_ctx = &per_cpu_ptr(pmu->pmu_cpu_context, dst_cpu)->ctx;
- mutex_lock(&src_ctx->mutex);
+ /*
+ * See perf_event_ctx_lock() for comments on the details
+ * of swizzling perf_event::ctx.
+ */
+ mutex_lock_double(&src_ctx->mutex, &dst_ctx->mutex);
list_for_each_entry_safe(event, tmp, &src_ctx->event_list,
event_entry) {
perf_remove_from_context(event, false);
put_ctx(src_ctx);
list_add(&event->migrate_entry, &events);
}
- mutex_unlock(&src_ctx->mutex);
+ /*
+ * Wait for the events to quiesce before re-instating them.
+ */
synchronize_rcu();
- mutex_lock(&dst_ctx->mutex);
+ /*
+ * Re-instate events in 2 passes.
+ *
+ * Skip over group leaders and only install siblings on this first
+ * pass, siblings will not get enabled without a leader, however a
+ * leader will enable its siblings, even if those are still on the old
+ * context.
+ */
+ list_for_each_entry_safe(event, tmp, &events, migrate_entry) {
+ if (event->group_leader == event)
+ continue;
+
+ list_del(&event->migrate_entry);
+ if (event->state >= PERF_EVENT_STATE_OFF)
+ event->state = PERF_EVENT_STATE_INACTIVE;
+ account_event_cpu(event, dst_cpu);
+ perf_install_in_context(dst_ctx, event, dst_cpu);
+ get_ctx(dst_ctx);
+ }
+
+ /*
+ * Once all the siblings are setup properly, install the group leaders
+ * to make it go.
+ */
list_for_each_entry_safe(event, tmp, &events, migrate_entry) {
list_del(&event->migrate_entry);
if (event->state >= PERF_EVENT_STATE_OFF)
get_ctx(dst_ctx);
}
mutex_unlock(&dst_ctx->mutex);
+ mutex_unlock(&src_ctx->mutex);
}
EXPORT_SYMBOL_GPL(perf_pmu_migrate_context);
put_event(parent);
+ raw_spin_lock_irq(&ctx->lock);
perf_group_detach(event);
list_del_event(event, ctx);
+ raw_spin_unlock_irq(&ctx->lock);
free_event(event);
}
/*
- * free an unexposed, unused context as created by inheritance by
+ * Free an unexposed, unused context as created by inheritance by
* perf_event_init_task below, used by fork() in case of fail.
+ *
+ * Not all locks are strictly required, but take them anyway to be nice and
+ * help out with the lockdep assertions.
*/
void perf_event_free_task(struct task_struct *task)
{
for_each_possible_cpu(cpu) {
swhash = &per_cpu(swevent_htable, cpu);
mutex_init(&swhash->hlist_mutex);
- INIT_LIST_HEAD(&per_cpu(rotation_list, cpu));
+ INIT_LIST_HEAD(&per_cpu(active_ctx_list, cpu));
}
}
}
#if defined CONFIG_HOTPLUG_CPU || defined CONFIG_KEXEC
-static void perf_pmu_rotate_stop(struct pmu *pmu)
-{
- struct perf_cpu_context *cpuctx = this_cpu_ptr(pmu->pmu_cpu_context);
-
- WARN_ON(!irqs_disabled());
-
- list_del_init(&cpuctx->rotation_list);
-}
-
static void __perf_event_exit_context(void *__info)
{
struct remove_event re = { .detach_group = true };
struct perf_event_context *ctx = __info;
- perf_pmu_rotate_stop(ctx->pmu);
-
rcu_read_lock();
list_for_each_entry_rcu(re.event, &ctx->event_list, event_entry)
__perf_remove_from_context(&re);
#include <linux/vmalloc.h>
#include <linux/slab.h>
#include <linux/circ_buf.h>
+#include <linux/poll.h>
#include "internal.h"
static void perf_output_wakeup(struct perf_output_handle *handle)
{
- atomic_set(&handle->rb->poll, POLL_IN);
+ atomic_set(&handle->rb->poll, POLLIN);
handle->event->pending_wakeup = 1;
irq_work_queue(&handle->event->pending);
* if there are waiters then it will block, it does PI, etc. (Due to
* races the kernel might see a 0 value of the futex too.)
*/
-static int futex_lock_pi(u32 __user *uaddr, unsigned int flags, int detect,
+static int futex_lock_pi(u32 __user *uaddr, unsigned int flags,
ktime_t *time, int trylock)
{
struct hrtimer_sleeper timeout, *to = NULL;
case FUTEX_WAKE_OP:
return futex_wake_op(uaddr, flags, uaddr2, val, val2, val3);
case FUTEX_LOCK_PI:
- return futex_lock_pi(uaddr, flags, val, timeout, 0);
+ return futex_lock_pi(uaddr, flags, timeout, 0);
case FUTEX_UNLOCK_PI:
return futex_unlock_pi(uaddr, flags);
case FUTEX_TRYLOCK_PI:
- return futex_lock_pi(uaddr, flags, 0, timeout, 1);
+ return futex_lock_pi(uaddr, flags, NULL, 1);
case FUTEX_WAIT_REQUEUE_PI:
val3 = FUTEX_BITSET_MATCH_ANY;
return futex_wait_requeue_pi(uaddr, flags, val, timeout, val3,
-obj-y += mutex.o semaphore.o rwsem.o mcs_spinlock.o
+obj-y += mutex.o semaphore.o rwsem.o
ifdef CONFIG_FUNCTION_TRACER
CFLAGS_REMOVE_lockdep.o = -pg
obj-$(CONFIG_LOCKDEP) += lockdep_proc.o
endif
obj-$(CONFIG_SMP) += spinlock.o
+obj-$(CONFIG_LOCK_SPIN_ON_OWNER) += osq_lock.o
obj-$(CONFIG_SMP) += lglock.o
obj-$(CONFIG_PROVE_LOCKING) += spinlock.o
obj-$(CONFIG_RT_MUTEXES) += rtmutex.o
arch_mcs_spin_unlock_contended(&next->locked);
}
-/*
- * Cancellable version of the MCS lock above.
- *
- * Intended for adaptive spinning of sleeping locks:
- * mutex_lock()/rwsem_down_{read,write}() etc.
- */
-
-struct optimistic_spin_node {
- struct optimistic_spin_node *next, *prev;
- int locked; /* 1 if lock acquired */
- int cpu; /* encoded CPU # value */
-};
-
-extern bool osq_lock(struct optimistic_spin_queue *lock);
-extern void osq_unlock(struct optimistic_spin_queue *lock);
-
#endif /* __LINUX_MCS_SPINLOCK_H */
* The mutex must later on be released by the same task that
* acquired it. Recursive locking is not allowed. The task
* may not exit without first unlocking the mutex. Also, kernel
- * memory where the mutex resides mutex must not be freed with
+ * memory where the mutex resides must not be freed with
* the mutex still locked. The mutex must first be initialized
* (or statically defined) before it can be locked. memset()-ing
* the mutex to 0 is not allowed.
}
/*
- * after acquiring lock with fastpath or when we lost out in contested
+ * After acquiring lock with fastpath or when we lost out in contested
* slowpath, set ctx and wake up any waiters so they can recheck.
*
* This function is never called when CONFIG_DEBUG_LOCK_ALLOC is set,
spin_unlock_mutex(&lock->base.wait_lock, flags);
}
-
-#ifdef CONFIG_MUTEX_SPIN_ON_OWNER
/*
- * In order to avoid a stampede of mutex spinners from acquiring the mutex
- * more or less simultaneously, the spinners need to acquire a MCS lock
- * first before spinning on the owner field.
+ * After acquiring lock in the slowpath set ctx and wake up any
+ * waiters so they can recheck.
*
+ * Callers must hold the mutex wait_lock.
*/
+static __always_inline void
+ww_mutex_set_context_slowpath(struct ww_mutex *lock,
+ struct ww_acquire_ctx *ctx)
+{
+ struct mutex_waiter *cur;
-/*
- * Mutex spinning code migrated from kernel/sched/core.c
- */
+ ww_mutex_lock_acquired(lock, ctx);
+ lock->ctx = ctx;
+
+ /*
+ * Give any possible sleeping processes the chance to wake up,
+ * so they can recheck if they have to back off.
+ */
+ list_for_each_entry(cur, &lock->base.wait_list, list) {
+ debug_mutex_wake_waiter(&lock->base, cur);
+ wake_up_process(cur->task);
+ }
+}
+#ifdef CONFIG_MUTEX_SPIN_ON_OWNER
static inline bool owner_running(struct mutex *lock, struct task_struct *owner)
{
if (lock->owner != owner)
if (!mutex_can_spin_on_owner(lock))
goto done;
+ /*
+ * In order to avoid a stampede of mutex spinners trying to
+ * acquire the mutex all at once, the spinners need to take a
+ * MCS (queued) lock first before spinning on the owner field.
+ */
if (!osq_lock(&lock->osq))
goto done;
EXPORT_SYMBOL(ww_mutex_unlock);
static inline int __sched
-__mutex_lock_check_stamp(struct mutex *lock, struct ww_acquire_ctx *ctx)
+__ww_mutex_lock_check_stamp(struct mutex *lock, struct ww_acquire_ctx *ctx)
{
struct ww_mutex *ww = container_of(lock, struct ww_mutex, base);
struct ww_acquire_ctx *hold_ctx = ACCESS_ONCE(ww->ctx);
}
if (use_ww_ctx && ww_ctx->acquired > 0) {
- ret = __mutex_lock_check_stamp(lock, ww_ctx);
+ ret = __ww_mutex_lock_check_stamp(lock, ww_ctx);
if (ret)
goto err;
}
schedule_preempt_disabled();
spin_lock_mutex(&lock->wait_lock, flags);
}
+ __set_task_state(task, TASK_RUNNING);
+
mutex_remove_waiter(lock, &waiter, current_thread_info());
/* set it to 0 if there are no waiters left: */
if (likely(list_empty(&lock->wait_list)))
if (use_ww_ctx) {
struct ww_mutex *ww = container_of(lock, struct ww_mutex, base);
- struct mutex_waiter *cur;
-
- /*
- * This branch gets optimized out for the common case,
- * and is only important for ww_mutex_lock.
- */
- ww_mutex_lock_acquired(ww, ww_ctx);
- ww->ctx = ww_ctx;
-
- /*
- * Give any possible sleeping processes the chance to wake up,
- * so they can recheck if they have to back off.
- */
- list_for_each_entry(cur, &lock->wait_list, list) {
- debug_mutex_wake_waiter(lock, cur);
- wake_up_process(cur->task);
- }
+ ww_mutex_set_context_slowpath(ww, ww_ctx);
}
spin_unlock_mutex(&lock->wait_lock, flags);
#include <linux/percpu.h>
#include <linux/sched.h>
-#include "mcs_spinlock.h"
-
-#ifdef CONFIG_SMP
+#include <linux/osq_lock.h>
/*
* An MCS like lock especially tailored for optimistic spinning for sleeping
* cmpxchg in an attempt to undo our queueing.
*/
- while (!smp_load_acquire(&node->locked)) {
+ while (!ACCESS_ONCE(node->locked)) {
/*
* If we need to reschedule bail... so we can block.
*/
if (next)
ACCESS_ONCE(next->locked) = 1;
}
-
-#endif
-
set_current_state(state);
}
+ __set_current_state(TASK_RUNNING);
return ret;
}
ret = task_blocks_on_rt_mutex(lock, &waiter, current, chwalk);
if (likely(!ret))
+ /* sleep on the mutex */
ret = __rt_mutex_slowlock(lock, state, timeout, &waiter);
- set_current_state(TASK_RUNNING);
-
if (unlikely(ret)) {
remove_waiter(lock, &waiter);
rt_mutex_handle_deadlock(ret, chwalk, &waiter);
set_current_state(TASK_INTERRUPTIBLE);
+ /* sleep on the mutex */
ret = __rt_mutex_slowlock(lock, TASK_INTERRUPTIBLE, to, waiter);
- set_current_state(TASK_RUNNING);
-
if (unlikely(ret))
remove_waiter(lock, waiter);
set_task_state(tsk, TASK_UNINTERRUPTIBLE);
}
- tsk->state = TASK_RUNNING;
+ __set_task_state(tsk, TASK_RUNNING);
out:
;
}
schedule();
}
- tsk->state = TASK_RUNNING;
-
+ __set_task_state(tsk, TASK_RUNNING);
return sem;
}
EXPORT_SYMBOL(rwsem_down_read_failed);
}
EXPORT_SYMBOL_GPL(raw_notifier_call_chain);
+#ifdef CONFIG_SRCU
/*
* SRCU notifier chain routines. Registration and unregistration
* use a mutex, and call_chain is synchronized by SRCU (no locks).
}
EXPORT_SYMBOL_GPL(srcu_init_notifier_head);
+#endif /* CONFIG_SRCU */
+
static ATOMIC_NOTIFIER_HEAD(die_chain);
int notrace notify_die(enum die_val val, const char *str,
config PM_OPP
bool
+ select SRCU
---help---
SOCs have a standard set of tuples consisting of frequency and
voltage pairs that the device will support per voltage domain. This
-obj-y += update.o srcu.o
+obj-y += update.o
+obj-$(CONFIG_SRCU) += srcu.o
obj-$(CONFIG_RCU_TORTURE_TEST) += rcutorture.o
obj-$(CONFIG_TREE_RCU) += tree.o
obj-$(CONFIG_PREEMPT_RCU) += tree.o
void rcu_early_boot_tests(void);
+/*
+ * This function really isn't for public consumption, but RCU is special in
+ * that context switches can allow the state machine to make progress.
+ */
+extern void resched_cpu(int cpu);
+
#endif /* __LINUX_RCU_H */
int (*readlock)(void);
void (*read_delay)(struct torture_random_state *rrsp);
void (*readunlock)(int idx);
- int (*completed)(void);
+ unsigned long (*started)(void);
+ unsigned long (*completed)(void);
void (*deferred_free)(struct rcu_torture *p);
void (*sync)(void);
void (*exp_sync)(void);
rcu_read_unlock();
}
-static int rcu_torture_completed(void)
-{
- return rcu_batches_completed();
-}
-
/*
* Update callback in the pipe. This should be invoked after a grace period.
*/
cur_ops->deferred_free(rp);
}
-static int rcu_no_completed(void)
+static unsigned long rcu_no_completed(void)
{
return 0;
}
.readlock = rcu_torture_read_lock,
.read_delay = rcu_read_delay,
.readunlock = rcu_torture_read_unlock,
- .completed = rcu_torture_completed,
+ .started = rcu_batches_started,
+ .completed = rcu_batches_completed,
.deferred_free = rcu_torture_deferred_free,
.sync = synchronize_rcu,
.exp_sync = synchronize_rcu_expedited,
rcu_read_unlock_bh();
}
-static int rcu_bh_torture_completed(void)
-{
- return rcu_batches_completed_bh();
-}
-
static void rcu_bh_torture_deferred_free(struct rcu_torture *p)
{
call_rcu_bh(&p->rtort_rcu, rcu_torture_cb);
.readlock = rcu_bh_torture_read_lock,
.read_delay = rcu_read_delay, /* just reuse rcu's version. */
.readunlock = rcu_bh_torture_read_unlock,
- .completed = rcu_bh_torture_completed,
+ .started = rcu_batches_started_bh,
+ .completed = rcu_batches_completed_bh,
.deferred_free = rcu_bh_torture_deferred_free,
.sync = synchronize_rcu_bh,
.exp_sync = synchronize_rcu_bh_expedited,
.readlock = rcu_torture_read_lock,
.read_delay = rcu_read_delay, /* just reuse rcu's version. */
.readunlock = rcu_torture_read_unlock,
+ .started = rcu_no_completed,
.completed = rcu_no_completed,
.deferred_free = rcu_busted_torture_deferred_free,
.sync = synchronize_rcu_busted,
srcu_read_unlock(&srcu_ctl, idx);
}
-static int srcu_torture_completed(void)
+static unsigned long srcu_torture_completed(void)
{
return srcu_batches_completed(&srcu_ctl);
}
.readlock = srcu_torture_read_lock,
.read_delay = srcu_read_delay,
.readunlock = srcu_torture_read_unlock,
+ .started = NULL,
.completed = srcu_torture_completed,
.deferred_free = srcu_torture_deferred_free,
.sync = srcu_torture_synchronize,
.readlock = sched_torture_read_lock,
.read_delay = rcu_read_delay, /* just reuse rcu's version. */
.readunlock = sched_torture_read_unlock,
- .completed = rcu_no_completed,
+ .started = rcu_batches_started_sched,
+ .completed = rcu_batches_completed_sched,
.deferred_free = rcu_sched_torture_deferred_free,
.sync = synchronize_sched,
.exp_sync = synchronize_sched_expedited,
.readlock = tasks_torture_read_lock,
.read_delay = rcu_read_delay, /* just reuse rcu's version. */
.readunlock = tasks_torture_read_unlock,
+ .started = rcu_no_completed,
.completed = rcu_no_completed,
.deferred_free = rcu_tasks_torture_deferred_free,
.sync = synchronize_rcu_tasks,
static void rcu_torture_timer(unsigned long unused)
{
int idx;
- int completed;
- int completed_end;
+ unsigned long started;
+ unsigned long completed;
static DEFINE_TORTURE_RANDOM(rand);
static DEFINE_SPINLOCK(rand_lock);
struct rcu_torture *p;
unsigned long long ts;
idx = cur_ops->readlock();
- completed = cur_ops->completed();
+ if (cur_ops->started)
+ started = cur_ops->started();
+ else
+ started = cur_ops->completed();
ts = rcu_trace_clock_local();
p = rcu_dereference_check(rcu_torture_current,
rcu_read_lock_bh_held() ||
/* Should not happen, but... */
pipe_count = RCU_TORTURE_PIPE_LEN;
}
- completed_end = cur_ops->completed();
+ completed = cur_ops->completed();
if (pipe_count > 1) {
do_trace_rcu_torture_read(cur_ops->name, &p->rtort_rcu, ts,
- completed, completed_end);
+ started, completed);
rcutorture_trace_dump();
}
__this_cpu_inc(rcu_torture_count[pipe_count]);
- completed = completed_end - completed;
+ completed = completed - started;
+ if (cur_ops->started)
+ completed++;
if (completed > RCU_TORTURE_PIPE_LEN) {
/* Should not happen, but... */
completed = RCU_TORTURE_PIPE_LEN;
static int
rcu_torture_reader(void *arg)
{
- int completed;
- int completed_end;
+ unsigned long started;
+ unsigned long completed;
int idx;
DEFINE_TORTURE_RANDOM(rand);
struct rcu_torture *p;
mod_timer(&t, jiffies + 1);
}
idx = cur_ops->readlock();
- completed = cur_ops->completed();
+ if (cur_ops->started)
+ started = cur_ops->started();
+ else
+ started = cur_ops->completed();
ts = rcu_trace_clock_local();
p = rcu_dereference_check(rcu_torture_current,
rcu_read_lock_bh_held() ||
/* Should not happen, but... */
pipe_count = RCU_TORTURE_PIPE_LEN;
}
- completed_end = cur_ops->completed();
+ completed = cur_ops->completed();
if (pipe_count > 1) {
do_trace_rcu_torture_read(cur_ops->name, &p->rtort_rcu,
- ts, completed, completed_end);
+ ts, started, completed);
rcutorture_trace_dump();
}
__this_cpu_inc(rcu_torture_count[pipe_count]);
- completed = completed_end - completed;
+ completed = completed - started;
+ if (cur_ops->started)
+ completed++;
if (completed > RCU_TORTURE_PIPE_LEN) {
/* Should not happen, but... */
completed = RCU_TORTURE_PIPE_LEN;
cur_ops->cb_barrier(); /* Implies smp_mb() for wait_event(). */
if (atomic_read(&barrier_cbs_invoked) != n_barrier_cbs) {
n_rcu_torture_barrier_error++;
+ pr_err("barrier_cbs_invoked = %d, n_barrier_cbs = %d\n",
+ atomic_read(&barrier_cbs_invoked),
+ n_barrier_cbs);
WARN_ON_ONCE(1);
}
n_barrier_successes++;
* Report the number of batches, correlated with, but not necessarily
* precisely the same as, the number of grace periods that have elapsed.
*/
-long srcu_batches_completed(struct srcu_struct *sp)
+unsigned long srcu_batches_completed(struct srcu_struct *sp)
{
return sp->completed;
}
void (*func)(struct rcu_head *rcu),
struct rcu_ctrlblk *rcp);
-static long long rcu_dynticks_nesting = DYNTICK_TASK_EXIT_IDLE;
-
#include "tiny_plugin.h"
-/* Common code for rcu_idle_enter() and rcu_irq_exit(), see kernel/rcu/tree.c. */
-static void rcu_idle_enter_common(long long newval)
-{
- if (newval) {
- RCU_TRACE(trace_rcu_dyntick(TPS("--="),
- rcu_dynticks_nesting, newval));
- rcu_dynticks_nesting = newval;
- return;
- }
- RCU_TRACE(trace_rcu_dyntick(TPS("Start"),
- rcu_dynticks_nesting, newval));
- if (IS_ENABLED(CONFIG_RCU_TRACE) && !is_idle_task(current)) {
- struct task_struct *idle __maybe_unused = idle_task(smp_processor_id());
-
- RCU_TRACE(trace_rcu_dyntick(TPS("Entry error: not idle task"),
- rcu_dynticks_nesting, newval));
- ftrace_dump(DUMP_ALL);
- WARN_ONCE(1, "Current pid: %d comm: %s / Idle pid: %d comm: %s",
- current->pid, current->comm,
- idle->pid, idle->comm); /* must be idle task! */
- }
- rcu_sched_qs(); /* implies rcu_bh_inc() */
- barrier();
- rcu_dynticks_nesting = newval;
-}
-
/*
* Enter idle, which is an extended quiescent state if we have fully
- * entered that mode (i.e., if the new value of dynticks_nesting is zero).
+ * entered that mode.
*/
void rcu_idle_enter(void)
{
- unsigned long flags;
- long long newval;
-
- local_irq_save(flags);
- WARN_ON_ONCE((rcu_dynticks_nesting & DYNTICK_TASK_NEST_MASK) == 0);
- if ((rcu_dynticks_nesting & DYNTICK_TASK_NEST_MASK) ==
- DYNTICK_TASK_NEST_VALUE)
- newval = 0;
- else
- newval = rcu_dynticks_nesting - DYNTICK_TASK_NEST_VALUE;
- rcu_idle_enter_common(newval);
- local_irq_restore(flags);
}
EXPORT_SYMBOL_GPL(rcu_idle_enter);
*/
void rcu_irq_exit(void)
{
- unsigned long flags;
- long long newval;
-
- local_irq_save(flags);
- newval = rcu_dynticks_nesting - 1;
- WARN_ON_ONCE(newval < 0);
- rcu_idle_enter_common(newval);
- local_irq_restore(flags);
}
EXPORT_SYMBOL_GPL(rcu_irq_exit);
-/* Common code for rcu_idle_exit() and rcu_irq_enter(), see kernel/rcu/tree.c. */
-static void rcu_idle_exit_common(long long oldval)
-{
- if (oldval) {
- RCU_TRACE(trace_rcu_dyntick(TPS("++="),
- oldval, rcu_dynticks_nesting));
- return;
- }
- RCU_TRACE(trace_rcu_dyntick(TPS("End"), oldval, rcu_dynticks_nesting));
- if (IS_ENABLED(CONFIG_RCU_TRACE) && !is_idle_task(current)) {
- struct task_struct *idle __maybe_unused = idle_task(smp_processor_id());
-
- RCU_TRACE(trace_rcu_dyntick(TPS("Exit error: not idle task"),
- oldval, rcu_dynticks_nesting));
- ftrace_dump(DUMP_ALL);
- WARN_ONCE(1, "Current pid: %d comm: %s / Idle pid: %d comm: %s",
- current->pid, current->comm,
- idle->pid, idle->comm); /* must be idle task! */
- }
-}
-
/*
* Exit idle, so that we are no longer in an extended quiescent state.
*/
void rcu_idle_exit(void)
{
- unsigned long flags;
- long long oldval;
-
- local_irq_save(flags);
- oldval = rcu_dynticks_nesting;
- WARN_ON_ONCE(rcu_dynticks_nesting < 0);
- if (rcu_dynticks_nesting & DYNTICK_TASK_NEST_MASK)
- rcu_dynticks_nesting += DYNTICK_TASK_NEST_VALUE;
- else
- rcu_dynticks_nesting = DYNTICK_TASK_EXIT_IDLE;
- rcu_idle_exit_common(oldval);
- local_irq_restore(flags);
}
EXPORT_SYMBOL_GPL(rcu_idle_exit);
*/
void rcu_irq_enter(void)
{
- unsigned long flags;
- long long oldval;
-
- local_irq_save(flags);
- oldval = rcu_dynticks_nesting;
- rcu_dynticks_nesting++;
- WARN_ON_ONCE(rcu_dynticks_nesting == 0);
- rcu_idle_exit_common(oldval);
- local_irq_restore(flags);
}
EXPORT_SYMBOL_GPL(rcu_irq_enter);
*/
bool notrace __rcu_is_watching(void)
{
- return rcu_dynticks_nesting;
+ return true;
}
EXPORT_SYMBOL(__rcu_is_watching);
#endif /* defined(CONFIG_DEBUG_LOCK_ALLOC) || defined(CONFIG_RCU_TRACE) */
-/*
- * Test whether the current CPU was interrupted from idle. Nested
- * interrupts don't count, we must be running at the first interrupt
- * level.
- */
-static int rcu_is_cpu_rrupt_from_idle(void)
-{
- return rcu_dynticks_nesting <= 1;
-}
-
/*
* Helper function for rcu_sched_qs() and rcu_bh_qs().
* Also irqs are disabled to avoid confusion due to interrupt handlers
void rcu_check_callbacks(int user)
{
RCU_TRACE(check_cpu_stalls());
- if (user || rcu_is_cpu_rrupt_from_idle())
+ if (user)
rcu_sched_qs();
else if (!in_softirq())
rcu_bh_qs();
rcp->curtail = &head->next;
RCU_TRACE(rcp->qlen++);
local_irq_restore(flags);
+
+ if (unlikely(is_idle_task(current))) {
+ /* force scheduling for rcu_sched_qs() */
+ resched_cpu(0);
+ }
}
/*
void __init rcu_init(void)
{
open_softirq(RCU_SOFTIRQ, rcu_process_callbacks);
+ RCU_TRACE(reset_cpu_stall_ticks(&rcu_sched_ctrlblk));
+ RCU_TRACE(reset_cpu_stall_ticks(&rcu_bh_ctrlblk));
rcu_early_boot_tests();
}
rcp->ticks_this_gp++;
j = jiffies;
js = ACCESS_ONCE(rcp->jiffies_stall);
- if (*rcp->curtail && ULONG_CMP_GE(j, js)) {
+ if (rcp->rcucblist && ULONG_CMP_GE(j, js)) {
pr_err("INFO: %s stall on CPU (%lu ticks this GP) idle=%llx (t=%lu jiffies q=%ld)\n",
- rcp->name, rcp->ticks_this_gp, rcu_dynticks_nesting,
+ rcp->name, rcp->ticks_this_gp, DYNTICK_TASK_EXIT_IDLE,
jiffies - rcp->gp_start, rcp->qlen);
dump_stack();
- }
- if (*rcp->curtail && ULONG_CMP_GE(j, js))
ACCESS_ONCE(rcp->jiffies_stall) = jiffies +
3 * rcu_jiffies_till_stall_check() + 3;
- else if (ULONG_CMP_GE(j, js))
+ } else if (ULONG_CMP_GE(j, js)) {
ACCESS_ONCE(rcp->jiffies_stall) = jiffies + rcu_jiffies_till_stall_check();
+ }
}
static void reset_cpu_stall_ticks(struct rcu_ctrlblk *rcp)
static void invoke_rcu_core(void);
static void invoke_rcu_callbacks(struct rcu_state *rsp, struct rcu_data *rdp);
+/* rcuc/rcub kthread realtime priority */
+static int kthread_prio = CONFIG_RCU_KTHREAD_PRIO;
+module_param(kthread_prio, int, 0644);
+
/*
* Track the rcutorture test sequence number and the update version
* number within a given test. The rcutorture_testseq is incremented
#endif /* #ifdef CONFIG_NO_HZ_FULL_SYSIDLE */
};
+DEFINE_PER_CPU_SHARED_ALIGNED(unsigned long, rcu_qs_ctr);
+EXPORT_PER_CPU_SYMBOL_GPL(rcu_qs_ctr);
+
/*
* Let the RCU core know that this CPU has gone through the scheduler,
* which is a quiescent state. This is called when the need for a
}
EXPORT_SYMBOL_GPL(rcu_note_context_switch);
+/*
+ * Register a quiesecent state for all RCU flavors. If there is an
+ * emergency, invoke rcu_momentary_dyntick_idle() to do a heavy-weight
+ * dyntick-idle quiescent state visible to other CPUs (but only for those
+ * RCU flavors in desparate need of a quiescent state, which will normally
+ * be none of them). Either way, do a lightweight quiescent state for
+ * all RCU flavors.
+ */
+void rcu_all_qs(void)
+{
+ if (unlikely(raw_cpu_read(rcu_sched_qs_mask)))
+ rcu_momentary_dyntick_idle();
+ this_cpu_inc(rcu_qs_ctr);
+}
+EXPORT_SYMBOL_GPL(rcu_all_qs);
+
static long blimit = 10; /* Maximum callbacks per rcu_do_batch. */
static long qhimark = 10000; /* If this many pending, ignore blimit. */
static long qlowmark = 100; /* Once only this many pending, use blimit. */
static int rcu_pending(void);
/*
- * Return the number of RCU-sched batches processed thus far for debug & stats.
+ * Return the number of RCU batches started thus far for debug & stats.
+ */
+unsigned long rcu_batches_started(void)
+{
+ return rcu_state_p->gpnum;
+}
+EXPORT_SYMBOL_GPL(rcu_batches_started);
+
+/*
+ * Return the number of RCU-sched batches started thus far for debug & stats.
+ */
+unsigned long rcu_batches_started_sched(void)
+{
+ return rcu_sched_state.gpnum;
+}
+EXPORT_SYMBOL_GPL(rcu_batches_started_sched);
+
+/*
+ * Return the number of RCU BH batches started thus far for debug & stats.
*/
-long rcu_batches_completed_sched(void)
+unsigned long rcu_batches_started_bh(void)
+{
+ return rcu_bh_state.gpnum;
+}
+EXPORT_SYMBOL_GPL(rcu_batches_started_bh);
+
+/*
+ * Return the number of RCU batches completed thus far for debug & stats.
+ */
+unsigned long rcu_batches_completed(void)
+{
+ return rcu_state_p->completed;
+}
+EXPORT_SYMBOL_GPL(rcu_batches_completed);
+
+/*
+ * Return the number of RCU-sched batches completed thus far for debug & stats.
+ */
+unsigned long rcu_batches_completed_sched(void)
{
return rcu_sched_state.completed;
}
EXPORT_SYMBOL_GPL(rcu_batches_completed_sched);
/*
- * Return the number of RCU BH batches processed thus far for debug & stats.
+ * Return the number of RCU BH batches completed thus far for debug & stats.
*/
-long rcu_batches_completed_bh(void)
+unsigned long rcu_batches_completed_bh(void)
{
return rcu_bh_state.completed;
}
/**
* rcu_nmi_enter - inform RCU of entry to NMI context
*
- * If the CPU was idle with dynamic ticks active, and there is no
- * irq handler running, this updates rdtp->dynticks_nmi to let the
- * RCU grace-period handling know that the CPU is active.
+ * If the CPU was idle from RCU's viewpoint, update rdtp->dynticks and
+ * rdtp->dynticks_nmi_nesting to let the RCU grace-period handling know
+ * that the CPU is active. This implementation permits nested NMIs, as
+ * long as the nesting level does not overflow an int. (You will probably
+ * run out of stack space first.)
*/
void rcu_nmi_enter(void)
{
struct rcu_dynticks *rdtp = this_cpu_ptr(&rcu_dynticks);
+ int incby = 2;
- if (rdtp->dynticks_nmi_nesting == 0 &&
- (atomic_read(&rdtp->dynticks) & 0x1))
- return;
- rdtp->dynticks_nmi_nesting++;
- smp_mb__before_atomic(); /* Force delay from prior write. */
- atomic_inc(&rdtp->dynticks);
- /* CPUs seeing atomic_inc() must see later RCU read-side crit sects */
- smp_mb__after_atomic(); /* See above. */
- WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1));
+ /* Complain about underflow. */
+ WARN_ON_ONCE(rdtp->dynticks_nmi_nesting < 0);
+
+ /*
+ * If idle from RCU viewpoint, atomically increment ->dynticks
+ * to mark non-idle and increment ->dynticks_nmi_nesting by one.
+ * Otherwise, increment ->dynticks_nmi_nesting by two. This means
+ * if ->dynticks_nmi_nesting is equal to one, we are guaranteed
+ * to be in the outermost NMI handler that interrupted an RCU-idle
+ * period (observation due to Andy Lutomirski).
+ */
+ if (!(atomic_read(&rdtp->dynticks) & 0x1)) {
+ smp_mb__before_atomic(); /* Force delay from prior write. */
+ atomic_inc(&rdtp->dynticks);
+ /* atomic_inc() before later RCU read-side crit sects */
+ smp_mb__after_atomic(); /* See above. */
+ WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1));
+ incby = 1;
+ }
+ rdtp->dynticks_nmi_nesting += incby;
+ barrier();
}
/**
* rcu_nmi_exit - inform RCU of exit from NMI context
*
- * If the CPU was idle with dynamic ticks active, and there is no
- * irq handler running, this updates rdtp->dynticks_nmi to let the
- * RCU grace-period handling know that the CPU is no longer active.
+ * If we are returning from the outermost NMI handler that interrupted an
+ * RCU-idle period, update rdtp->dynticks and rdtp->dynticks_nmi_nesting
+ * to let the RCU grace-period handling know that the CPU is back to
+ * being RCU-idle.
*/
void rcu_nmi_exit(void)
{
struct rcu_dynticks *rdtp = this_cpu_ptr(&rcu_dynticks);
- if (rdtp->dynticks_nmi_nesting == 0 ||
- --rdtp->dynticks_nmi_nesting != 0)
+ /*
+ * Check for ->dynticks_nmi_nesting underflow and bad ->dynticks.
+ * (We are exiting an NMI handler, so RCU better be paying attention
+ * to us!)
+ */
+ WARN_ON_ONCE(rdtp->dynticks_nmi_nesting <= 0);
+ WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1));
+
+ /*
+ * If the nesting level is not 1, the CPU wasn't RCU-idle, so
+ * leave it in non-RCU-idle state.
+ */
+ if (rdtp->dynticks_nmi_nesting != 1) {
+ rdtp->dynticks_nmi_nesting -= 2;
return;
+ }
+
+ /* This NMI interrupted an RCU-idle CPU, restore RCU-idleness. */
+ rdtp->dynticks_nmi_nesting = 0;
/* CPUs seeing atomic_inc() must see prior RCU read-side crit sects */
smp_mb__before_atomic(); /* See above. */
atomic_inc(&rdtp->dynticks);
trace_rcu_fqs(rdp->rsp->name, rdp->gpnum, rdp->cpu, TPS("dti"));
return 1;
} else {
+ if (ULONG_CMP_LT(ACCESS_ONCE(rdp->gpnum) + ULONG_MAX / 4,
+ rdp->mynode->gpnum))
+ ACCESS_ONCE(rdp->gpwrap) = true;
return 0;
}
}
-/*
- * This function really isn't for public consumption, but RCU is special in
- * that context switches can allow the state machine to make progress.
- */
-extern void resched_cpu(int cpu);
-
/*
* Return true if the specified CPU has passed through a quiescent
* state by virtue of being in or having passed through an dynticks
j1 = rcu_jiffies_till_stall_check();
ACCESS_ONCE(rsp->jiffies_stall) = j + j1;
rsp->jiffies_resched = j + j1 / 2;
+ rsp->n_force_qs_gpstart = ACCESS_ONCE(rsp->n_force_qs);
+}
+
+/*
+ * Complain about starvation of grace-period kthread.
+ */
+static void rcu_check_gp_kthread_starvation(struct rcu_state *rsp)
+{
+ unsigned long gpa;
+ unsigned long j;
+
+ j = jiffies;
+ gpa = ACCESS_ONCE(rsp->gp_activity);
+ if (j - gpa > 2 * HZ)
+ pr_err("%s kthread starved for %ld jiffies!\n",
+ rsp->name, j - gpa);
}
/*
}
}
-static void print_other_cpu_stall(struct rcu_state *rsp)
+static void print_other_cpu_stall(struct rcu_state *rsp, unsigned long gpnum)
{
int cpu;
long delta;
unsigned long flags;
+ unsigned long gpa;
+ unsigned long j;
int ndetected = 0;
struct rcu_node *rnp = rcu_get_root(rsp);
long totqlen = 0;
raw_spin_unlock_irqrestore(&rnp->lock, flags);
}
- /*
- * Now rat on any tasks that got kicked up to the root rcu_node
- * due to CPU offlining.
- */
- rnp = rcu_get_root(rsp);
- raw_spin_lock_irqsave(&rnp->lock, flags);
- ndetected += rcu_print_task_stall(rnp);
- raw_spin_unlock_irqrestore(&rnp->lock, flags);
-
print_cpu_stall_info_end();
for_each_possible_cpu(cpu)
totqlen += per_cpu_ptr(rsp->rda, cpu)->qlen;
pr_cont("(detected by %d, t=%ld jiffies, g=%ld, c=%ld, q=%lu)\n",
smp_processor_id(), (long)(jiffies - rsp->gp_start),
(long)rsp->gpnum, (long)rsp->completed, totqlen);
- if (ndetected == 0)
- pr_err("INFO: Stall ended before state dump start\n");
- else
+ if (ndetected) {
rcu_dump_cpu_stacks(rsp);
+ } else {
+ if (ACCESS_ONCE(rsp->gpnum) != gpnum ||
+ ACCESS_ONCE(rsp->completed) == gpnum) {
+ pr_err("INFO: Stall ended before state dump start\n");
+ } else {
+ j = jiffies;
+ gpa = ACCESS_ONCE(rsp->gp_activity);
+ pr_err("All QSes seen, last %s kthread activity %ld (%ld-%ld), jiffies_till_next_fqs=%ld\n",
+ rsp->name, j - gpa, j, gpa,
+ jiffies_till_next_fqs);
+ /* In this case, the current CPU might be at fault. */
+ sched_show_task(current);
+ }
+ }
/* Complain about tasks blocking the grace period. */
-
rcu_print_detail_task_stall(rsp);
+ rcu_check_gp_kthread_starvation(rsp);
+
force_quiescent_state(rsp); /* Kick them all. */
}
pr_cont(" (t=%lu jiffies g=%ld c=%ld q=%lu)\n",
jiffies - rsp->gp_start,
(long)rsp->gpnum, (long)rsp->completed, totqlen);
+
+ rcu_check_gp_kthread_starvation(rsp);
+
rcu_dump_cpu_stacks(rsp);
raw_spin_lock_irqsave(&rnp->lock, flags);
ULONG_CMP_GE(j, js + RCU_STALL_RAT_DELAY)) {
/* They had a few time units to dump stack, so complain. */
- print_other_cpu_stall(rsp);
+ print_other_cpu_stall(rsp, gpnum);
}
}
bool ret;
/* Handle the ends of any preceding grace periods first. */
- if (rdp->completed == rnp->completed) {
+ if (rdp->completed == rnp->completed &&
+ !unlikely(ACCESS_ONCE(rdp->gpwrap))) {
/* No grace period end, so just accelerate recent callbacks. */
ret = rcu_accelerate_cbs(rsp, rnp, rdp);
trace_rcu_grace_period(rsp->name, rdp->gpnum, TPS("cpuend"));
}
- if (rdp->gpnum != rnp->gpnum) {
+ if (rdp->gpnum != rnp->gpnum || unlikely(ACCESS_ONCE(rdp->gpwrap))) {
/*
* If the current grace period is waiting for this CPU,
* set up to detect a quiescent state, otherwise don't
rdp->gpnum = rnp->gpnum;
trace_rcu_grace_period(rsp->name, rdp->gpnum, TPS("cpustart"));
rdp->passed_quiesce = 0;
+ rdp->rcu_qs_ctr_snap = __this_cpu_read(rcu_qs_ctr);
rdp->qs_pending = !!(rnp->qsmask & rdp->grpmask);
zero_cpu_stall_ticks(rdp);
+ ACCESS_ONCE(rdp->gpwrap) = false;
}
return ret;
}
local_irq_save(flags);
rnp = rdp->mynode;
if ((rdp->gpnum == ACCESS_ONCE(rnp->gpnum) &&
- rdp->completed == ACCESS_ONCE(rnp->completed)) || /* w/out lock. */
+ rdp->completed == ACCESS_ONCE(rnp->completed) &&
+ !unlikely(ACCESS_ONCE(rdp->gpwrap))) || /* w/out lock. */
!raw_spin_trylock(&rnp->lock)) { /* irqs already off, so later. */
local_irq_restore(flags);
return;
struct rcu_data *rdp;
struct rcu_node *rnp = rcu_get_root(rsp);
+ ACCESS_ONCE(rsp->gp_activity) = jiffies;
rcu_bind_gp_kthread();
raw_spin_lock_irq(&rnp->lock);
smp_mb__after_unlock_lock();
rnp->grphi, rnp->qsmask);
raw_spin_unlock_irq(&rnp->lock);
cond_resched_rcu_qs();
+ ACCESS_ONCE(rsp->gp_activity) = jiffies;
}
mutex_unlock(&rsp->onoff_mutex);
unsigned long maxj;
struct rcu_node *rnp = rcu_get_root(rsp);
+ ACCESS_ONCE(rsp->gp_activity) = jiffies;
rsp->n_force_qs++;
if (fqs_state == RCU_SAVE_DYNTICK) {
/* Collect dyntick-idle snapshots. */
struct rcu_data *rdp;
struct rcu_node *rnp = rcu_get_root(rsp);
+ ACCESS_ONCE(rsp->gp_activity) = jiffies;
raw_spin_lock_irq(&rnp->lock);
smp_mb__after_unlock_lock();
gp_duration = jiffies - rsp->gp_start;
nocb += rcu_future_gp_cleanup(rsp, rnp);
raw_spin_unlock_irq(&rnp->lock);
cond_resched_rcu_qs();
+ ACCESS_ONCE(rsp->gp_activity) = jiffies;
}
rnp = rcu_get_root(rsp);
raw_spin_lock_irq(&rnp->lock);
if (rcu_gp_init(rsp))
break;
cond_resched_rcu_qs();
+ ACCESS_ONCE(rsp->gp_activity) = jiffies;
WARN_ON(signal_pending(current));
trace_rcu_grace_period(rsp->name,
ACCESS_ONCE(rsp->gpnum),
ACCESS_ONCE(rsp->gpnum),
TPS("fqsend"));
cond_resched_rcu_qs();
+ ACCESS_ONCE(rsp->gp_activity) = jiffies;
} else {
/* Deal with stray signal. */
cond_resched_rcu_qs();
+ ACCESS_ONCE(rsp->gp_activity) = jiffies;
WARN_ON(signal_pending(current));
trace_rcu_grace_period(rsp->name,
ACCESS_ONCE(rsp->gpnum),
rnp = rdp->mynode;
raw_spin_lock_irqsave(&rnp->lock, flags);
smp_mb__after_unlock_lock();
- if (rdp->passed_quiesce == 0 || rdp->gpnum != rnp->gpnum ||
- rnp->completed == rnp->gpnum) {
+ if ((rdp->passed_quiesce == 0 &&
+ rdp->rcu_qs_ctr_snap == __this_cpu_read(rcu_qs_ctr)) ||
+ rdp->gpnum != rnp->gpnum || rnp->completed == rnp->gpnum ||
+ rdp->gpwrap) {
/*
* The grace period in which this quiescent state was
* within the current grace period.
*/
rdp->passed_quiesce = 0; /* need qs for new gp. */
+ rdp->rcu_qs_ctr_snap = __this_cpu_read(rcu_qs_ctr);
raw_spin_unlock_irqrestore(&rnp->lock, flags);
return;
}
* Was there a quiescent state since the beginning of the grace
* period? If no, then exit and wait for the next call.
*/
- if (!rdp->passed_quiesce)
+ if (!rdp->passed_quiesce &&
+ rdp->rcu_qs_ctr_snap == __this_cpu_read(rcu_qs_ctr))
return;
/*
TPS("cpuofl"));
}
+/*
+ * All CPUs for the specified rcu_node structure have gone offline,
+ * and all tasks that were preempted within an RCU read-side critical
+ * section while running on one of those CPUs have since exited their RCU
+ * read-side critical section. Some other CPU is reporting this fact with
+ * the specified rcu_node structure's ->lock held and interrupts disabled.
+ * This function therefore goes up the tree of rcu_node structures,
+ * clearing the corresponding bits in the ->qsmaskinit fields. Note that
+ * the leaf rcu_node structure's ->qsmaskinit field has already been
+ * updated
+ *
+ * This function does check that the specified rcu_node structure has
+ * all CPUs offline and no blocked tasks, so it is OK to invoke it
+ * prematurely. That said, invoking it after the fact will cost you
+ * a needless lock acquisition. So once it has done its work, don't
+ * invoke it again.
+ */
+static void rcu_cleanup_dead_rnp(struct rcu_node *rnp_leaf)
+{
+ long mask;
+ struct rcu_node *rnp = rnp_leaf;
+
+ if (rnp->qsmaskinit || rcu_preempt_has_tasks(rnp))
+ return;
+ for (;;) {
+ mask = rnp->grpmask;
+ rnp = rnp->parent;
+ if (!rnp)
+ break;
+ raw_spin_lock(&rnp->lock); /* irqs already disabled. */
+ smp_mb__after_unlock_lock(); /* GP memory ordering. */
+ rnp->qsmaskinit &= ~mask;
+ if (rnp->qsmaskinit) {
+ raw_spin_unlock(&rnp->lock); /* irqs remain disabled. */
+ return;
+ }
+ raw_spin_unlock(&rnp->lock); /* irqs remain disabled. */
+ }
+}
+
/*
* The CPU has been completely removed, and some other CPU is reporting
* this fact from process context. Do the remainder of the cleanup,
static void rcu_cleanup_dead_cpu(int cpu, struct rcu_state *rsp)
{
unsigned long flags;
- unsigned long mask;
- int need_report = 0;
struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
struct rcu_node *rnp = rdp->mynode; /* Outgoing CPU's rdp & rnp. */
/* Orphan the dead CPU's callbacks, and adopt them if appropriate. */
rcu_send_cbs_to_orphanage(cpu, rsp, rnp, rdp);
rcu_adopt_orphan_cbs(rsp, flags);
+ raw_spin_unlock_irqrestore(&rsp->orphan_lock, flags);
- /* Remove the outgoing CPU from the masks in the rcu_node hierarchy. */
- mask = rdp->grpmask; /* rnp->grplo is constant. */
- do {
- raw_spin_lock(&rnp->lock); /* irqs already disabled. */
- smp_mb__after_unlock_lock();
- rnp->qsmaskinit &= ~mask;
- if (rnp->qsmaskinit != 0) {
- if (rnp != rdp->mynode)
- raw_spin_unlock(&rnp->lock); /* irqs remain disabled. */
- break;
- }
- if (rnp == rdp->mynode)
- need_report = rcu_preempt_offline_tasks(rsp, rnp, rdp);
- else
- raw_spin_unlock(&rnp->lock); /* irqs remain disabled. */
- mask = rnp->grpmask;
- rnp = rnp->parent;
- } while (rnp != NULL);
-
- /*
- * We still hold the leaf rcu_node structure lock here, and
- * irqs are still disabled. The reason for this subterfuge is
- * because invoking rcu_report_unblock_qs_rnp() with ->orphan_lock
- * held leads to deadlock.
- */
- raw_spin_unlock(&rsp->orphan_lock); /* irqs remain disabled. */
- rnp = rdp->mynode;
- if (need_report & RCU_OFL_TASKS_NORM_GP)
- rcu_report_unblock_qs_rnp(rnp, flags);
- else
- raw_spin_unlock_irqrestore(&rnp->lock, flags);
- if (need_report & RCU_OFL_TASKS_EXP_GP)
- rcu_report_exp_rnp(rsp, rnp, true);
+ /* Remove outgoing CPU from mask in the leaf rcu_node structure. */
+ raw_spin_lock_irqsave(&rnp->lock, flags);
+ smp_mb__after_unlock_lock(); /* Enforce GP memory-order guarantee. */
+ rnp->qsmaskinit &= ~rdp->grpmask;
+ if (rnp->qsmaskinit == 0 && !rcu_preempt_has_tasks(rnp))
+ rcu_cleanup_dead_rnp(rnp);
+ rcu_report_qs_rnp(rdp->grpmask, rsp, rnp, flags); /* Rlses rnp->lock. */
WARN_ONCE(rdp->qlen != 0 || rdp->nxtlist != NULL,
"rcu_cleanup_dead_cpu: Callbacks on offline CPU %d: qlen=%lu, nxtlist=%p\n",
cpu, rdp->qlen, rdp->nxtlist);
{
}
+static void __maybe_unused rcu_cleanup_dead_rnp(struct rcu_node *rnp_leaf)
+{
+}
+
static void rcu_cleanup_dead_cpu(int cpu, struct rcu_state *rsp)
{
}
}
raw_spin_unlock_irqrestore(&rnp->lock, flags);
}
- rnp = rcu_get_root(rsp);
- if (rnp->qsmask == 0) {
- raw_spin_lock_irqsave(&rnp->lock, flags);
- smp_mb__after_unlock_lock();
- rcu_initiate_boost(rnp, flags); /* releases rnp->lock. */
- }
}
/*
* Schedule RCU callback invocation. If the specified type of RCU
* does not support RCU priority boosting, just do a direct call,
* otherwise wake up the per-CPU kernel kthread. Note that because we
- * are running on the current CPU with interrupts disabled, the
+ * are running on the current CPU with softirqs disabled, the
* rcu_cpu_kthread_task cannot disappear out from under us.
*/
static void invoke_rcu_callbacks(struct rcu_state *rsp, struct rcu_data *rdp)
/* Is the RCU core waiting for a quiescent state from this CPU? */
if (rcu_scheduler_fully_active &&
- rdp->qs_pending && !rdp->passed_quiesce) {
+ rdp->qs_pending && !rdp->passed_quiesce &&
+ rdp->rcu_qs_ctr_snap == __this_cpu_read(rcu_qs_ctr)) {
rdp->n_rp_qs_pending++;
- } else if (rdp->qs_pending && rdp->passed_quiesce) {
+ } else if (rdp->qs_pending &&
+ (rdp->passed_quiesce ||
+ rdp->rcu_qs_ctr_snap != __this_cpu_read(rcu_qs_ctr))) {
rdp->n_rp_report_qs++;
return 1;
}
}
/* Has a new RCU grace period started? */
- if (ACCESS_ONCE(rnp->gpnum) != rdp->gpnum) { /* outside lock */
+ if (ACCESS_ONCE(rnp->gpnum) != rdp->gpnum ||
+ unlikely(ACCESS_ONCE(rdp->gpwrap))) { /* outside lock */
rdp->n_rp_gp_started++;
return 1;
}
} else {
_rcu_barrier_trace(rsp, "OnlineNoCB", cpu,
rsp->n_barrier_done);
+ smp_mb__before_atomic();
atomic_inc(&rsp->barrier_cpu_count);
__call_rcu(&rdp->barrier_head,
rcu_barrier_callback, rsp, cpu, 0);
/* Set up local state, ensuring consistent view of global state. */
raw_spin_lock_irqsave(&rnp->lock, flags);
rdp->grpmask = 1UL << (cpu - rdp->mynode->grplo);
- init_callback_list(rdp);
- rdp->qlen_lazy = 0;
- ACCESS_ONCE(rdp->qlen) = 0;
rdp->dynticks = &per_cpu(rcu_dynticks, cpu);
WARN_ON_ONCE(rdp->dynticks->dynticks_nesting != DYNTICK_TASK_EXIT_IDLE);
WARN_ON_ONCE(atomic_read(&rdp->dynticks->dynticks) != 1);
rdp->gpnum = rnp->completed;
rdp->completed = rnp->completed;
rdp->passed_quiesce = 0;
+ rdp->rcu_qs_ctr_snap = __this_cpu_read(rcu_qs_ctr);
rdp->qs_pending = 0;
trace_rcu_grace_period(rsp->name, rdp->gpnum, TPS("cpuonl"));
}
static int __init rcu_spawn_gp_kthread(void)
{
unsigned long flags;
+ int kthread_prio_in = kthread_prio;
struct rcu_node *rnp;
struct rcu_state *rsp;
+ struct sched_param sp;
struct task_struct *t;
+ /* Force priority into range. */
+ if (IS_ENABLED(CONFIG_RCU_BOOST) && kthread_prio < 1)
+ kthread_prio = 1;
+ else if (kthread_prio < 0)
+ kthread_prio = 0;
+ else if (kthread_prio > 99)
+ kthread_prio = 99;
+ if (kthread_prio != kthread_prio_in)
+ pr_alert("rcu_spawn_gp_kthread(): Limited prio to %d from %d\n",
+ kthread_prio, kthread_prio_in);
+
rcu_scheduler_fully_active = 1;
for_each_rcu_flavor(rsp) {
- t = kthread_run(rcu_gp_kthread, rsp, "%s", rsp->name);
+ t = kthread_create(rcu_gp_kthread, rsp, "%s", rsp->name);
BUG_ON(IS_ERR(t));
rnp = rcu_get_root(rsp);
raw_spin_lock_irqsave(&rnp->lock, flags);
rsp->gp_kthread = t;
+ if (kthread_prio) {
+ sp.sched_priority = kthread_prio;
+ sched_setscheduler_nocheck(t, SCHED_FIFO, &sp);
+ }
+ wake_up_process(t);
raw_spin_unlock_irqrestore(&rnp->lock, flags);
}
rcu_spawn_nocb_kthreads();
#include <linux/threads.h>
#include <linux/cpumask.h>
#include <linux/seqlock.h>
-#include <linux/irq_work.h>
/*
* Define shape of hierarchy based on NR_CPUS, CONFIG_RCU_FANOUT, and
/* queued on this rcu_node structure that */
/* are blocking the current grace period, */
/* there can be no such task. */
- struct completion boost_completion;
- /* Used to ensure that the rt_mutex used */
- /* to carry out the boosting is fully */
- /* released with no future boostee accesses */
- /* before that rt_mutex is re-initialized. */
struct rt_mutex boost_mtx;
/* Used only for the priority-boosting */
/* side effect, not as a lock. */
/* in order to detect GP end. */
unsigned long gpnum; /* Highest gp number that this CPU */
/* is aware of having started. */
+ unsigned long rcu_qs_ctr_snap;/* Snapshot of rcu_qs_ctr to check */
+ /* for rcu_all_qs() invocations. */
bool passed_quiesce; /* User-mode/idle loop etc. */
bool qs_pending; /* Core waits for quiesc state. */
bool beenonline; /* CPU online at least once. */
+ bool gpwrap; /* Possible gpnum/completed wrap. */
struct rcu_node *mynode; /* This CPU's leaf of hierarchy */
unsigned long grpmask; /* Mask to apply to leaf qsmask. */
#ifdef CONFIG_RCU_CPU_STALL_INFO
#ifdef CONFIG_RCU_NOCB_CPU
struct rcu_head *nocb_head; /* CBs waiting for kthread. */
struct rcu_head **nocb_tail;
- atomic_long_t nocb_q_count; /* # CBs waiting for kthread */
- atomic_long_t nocb_q_count_lazy; /* (approximate). */
+ atomic_long_t nocb_q_count; /* # CBs waiting for nocb */
+ atomic_long_t nocb_q_count_lazy; /* invocation (all stages). */
struct rcu_head *nocb_follower_head; /* CBs ready to invoke. */
struct rcu_head **nocb_follower_tail;
- atomic_long_t nocb_follower_count; /* # CBs ready to invoke. */
- atomic_long_t nocb_follower_count_lazy; /* (approximate). */
- int nocb_p_count; /* # CBs being invoked by kthread */
- int nocb_p_count_lazy; /* (approximate). */
wait_queue_head_t nocb_wq; /* For nocb kthreads to sleep on. */
struct task_struct *nocb_kthread;
int nocb_defer_wakeup; /* Defer wakeup of nocb_kthread. */
struct rcu_head *nocb_gp_head ____cacheline_internodealigned_in_smp;
/* CBs waiting for GP. */
struct rcu_head **nocb_gp_tail;
- long nocb_gp_count;
- long nocb_gp_count_lazy;
bool nocb_leader_sleep; /* Is the nocb leader thread asleep? */
struct rcu_data *nocb_next_follower;
/* Next follower in wakeup chain. */
/* due to no GP active. */
unsigned long gp_start; /* Time at which GP started, */
/* but in jiffies. */
+ unsigned long gp_activity; /* Time of last GP kthread */
+ /* activity in jiffies. */
unsigned long jiffies_stall; /* Time at which to check */
/* for CPU stalls. */
unsigned long jiffies_resched; /* Time at which to resched */
/* a reluctant CPU. */
+ unsigned long n_force_qs_gpstart; /* Snapshot of n_force_qs at */
+ /* GP start. */
unsigned long gp_max; /* Maximum GP duration in */
/* jiffies. */
const char *name; /* Name of structure. */
#define for_each_rcu_flavor(rsp) \
list_for_each_entry((rsp), &rcu_struct_flavors, flavors)
-/* Return values for rcu_preempt_offline_tasks(). */
-
-#define RCU_OFL_TASKS_NORM_GP 0x1 /* Tasks blocking normal */
- /* GP were moved to root. */
-#define RCU_OFL_TASKS_EXP_GP 0x2 /* Tasks blocking expedited */
- /* GP were moved to root. */
-
/*
* RCU implementation internal declarations:
*/
/* Forward declarations for rcutree_plugin.h */
static void rcu_bootup_announce(void);
-long rcu_batches_completed(void);
static void rcu_preempt_note_context_switch(void);
static int rcu_preempt_blocked_readers_cgp(struct rcu_node *rnp);
#ifdef CONFIG_HOTPLUG_CPU
-static void rcu_report_unblock_qs_rnp(struct rcu_node *rnp,
- unsigned long flags);
+static bool rcu_preempt_has_tasks(struct rcu_node *rnp);
#endif /* #ifdef CONFIG_HOTPLUG_CPU */
static void rcu_print_detail_task_stall(struct rcu_state *rsp);
static int rcu_print_task_stall(struct rcu_node *rnp);
static void rcu_preempt_check_blocked_tasks(struct rcu_node *rnp);
-#ifdef CONFIG_HOTPLUG_CPU
-static int rcu_preempt_offline_tasks(struct rcu_state *rsp,
- struct rcu_node *rnp,
- struct rcu_data *rdp);
-#endif /* #ifdef CONFIG_HOTPLUG_CPU */
static void rcu_preempt_check_callbacks(void);
void call_rcu(struct rcu_head *head, void (*func)(struct rcu_head *rcu));
-#if defined(CONFIG_HOTPLUG_CPU) || defined(CONFIG_PREEMPT_RCU)
-static void rcu_report_exp_rnp(struct rcu_state *rsp, struct rcu_node *rnp,
- bool wake);
-#endif /* #if defined(CONFIG_HOTPLUG_CPU) || defined(CONFIG_PREEMPT_RCU) */
static void __init __rcu_init_preempt(void);
static void rcu_initiate_boost(struct rcu_node *rnp, unsigned long flags);
static void rcu_preempt_boost_start_gp(struct rcu_node *rnp);
#endif /* #ifndef RCU_TREE_NONCORE */
#ifdef CONFIG_RCU_TRACE
-#ifdef CONFIG_RCU_NOCB_CPU
-/* Sum up queue lengths for tracing. */
+/* Read out queue lengths for tracing. */
static inline void rcu_nocb_q_lengths(struct rcu_data *rdp, long *ql, long *qll)
{
- *ql = atomic_long_read(&rdp->nocb_q_count) +
- rdp->nocb_p_count +
- atomic_long_read(&rdp->nocb_follower_count) +
- rdp->nocb_p_count + rdp->nocb_gp_count;
- *qll = atomic_long_read(&rdp->nocb_q_count_lazy) +
- rdp->nocb_p_count_lazy +
- atomic_long_read(&rdp->nocb_follower_count_lazy) +
- rdp->nocb_p_count_lazy + rdp->nocb_gp_count_lazy;
-}
+#ifdef CONFIG_RCU_NOCB_CPU
+ *ql = atomic_long_read(&rdp->nocb_q_count);
+ *qll = atomic_long_read(&rdp->nocb_q_count_lazy);
#else /* #ifdef CONFIG_RCU_NOCB_CPU */
-static inline void rcu_nocb_q_lengths(struct rcu_data *rdp, long *ql, long *qll)
-{
*ql = 0;
*qll = 0;
-}
#endif /* #else #ifdef CONFIG_RCU_NOCB_CPU */
+}
#endif /* #ifdef CONFIG_RCU_TRACE */
#include "../locking/rtmutex_common.h"
-/* rcuc/rcub kthread realtime priority */
-static int kthread_prio = CONFIG_RCU_KTHREAD_PRIO;
-module_param(kthread_prio, int, 0644);
-
/*
* Control variables for per-CPU and per-rcu_node kthreads. These
* handle all flavors of RCU.
static struct rcu_state *rcu_state_p = &rcu_preempt_state;
static int rcu_preempted_readers_exp(struct rcu_node *rnp);
+static void rcu_report_exp_rnp(struct rcu_state *rsp, struct rcu_node *rnp,
+ bool wake);
/*
* Tell them what RCU they are running.
rcu_bootup_announce_oddness();
}
-/*
- * Return the number of RCU-preempt batches processed thus far
- * for debug and statistics.
- */
-static long rcu_batches_completed_preempt(void)
-{
- return rcu_preempt_state.completed;
-}
-EXPORT_SYMBOL_GPL(rcu_batches_completed_preempt);
-
-/*
- * Return the number of RCU batches processed thus far for debug & stats.
- */
-long rcu_batches_completed(void)
-{
- return rcu_batches_completed_preempt();
-}
-EXPORT_SYMBOL_GPL(rcu_batches_completed);
-
/*
* Record a preemptible-RCU quiescent state for the specified CPU. Note
* that this just means that the task currently running on the CPU is
return np;
}
+/*
+ * Return true if the specified rcu_node structure has tasks that were
+ * preempted within an RCU read-side critical section.
+ */
+static bool rcu_preempt_has_tasks(struct rcu_node *rnp)
+{
+ return !list_empty(&rnp->blkd_tasks);
+}
+
/*
* Handle special cases during rcu_read_unlock(), such as needing to
* notify RCU core processing or task having blocked during the RCU
*/
void rcu_read_unlock_special(struct task_struct *t)
{
- int empty;
- int empty_exp;
- int empty_exp_now;
+ bool empty;
+ bool empty_exp;
+ bool empty_norm;
+ bool empty_exp_now;
unsigned long flags;
struct list_head *np;
#ifdef CONFIG_RCU_BOOST
break;
raw_spin_unlock(&rnp->lock); /* irqs remain disabled. */
}
- empty = !rcu_preempt_blocked_readers_cgp(rnp);
+ empty = !rcu_preempt_has_tasks(rnp);
+ empty_norm = !rcu_preempt_blocked_readers_cgp(rnp);
empty_exp = !rcu_preempted_readers_exp(rnp);
smp_mb(); /* ensure expedited fastpath sees end of RCU c-s. */
np = rcu_next_node_entry(t, rnp);
drop_boost_mutex = rt_mutex_owner(&rnp->boost_mtx) == t;
#endif /* #ifdef CONFIG_RCU_BOOST */
+ /*
+ * If this was the last task on the list, go see if we
+ * need to propagate ->qsmaskinit bit clearing up the
+ * rcu_node tree.
+ */
+ if (!empty && !rcu_preempt_has_tasks(rnp))
+ rcu_cleanup_dead_rnp(rnp);
+
/*
* If this was the last task on the current list, and if
* we aren't waiting on any CPUs, report the quiescent state.
* so we must take a snapshot of the expedited state.
*/
empty_exp_now = !rcu_preempted_readers_exp(rnp);
- if (!empty && !rcu_preempt_blocked_readers_cgp(rnp)) {
+ if (!empty_norm && !rcu_preempt_blocked_readers_cgp(rnp)) {
trace_rcu_quiescent_state_report(TPS("preempt_rcu"),
rnp->gpnum,
0, rnp->qsmask,
#ifdef CONFIG_RCU_BOOST
/* Unboost if we were boosted. */
- if (drop_boost_mutex) {
+ if (drop_boost_mutex)
rt_mutex_unlock(&rnp->boost_mtx);
- complete(&rnp->boost_completion);
- }
#endif /* #ifdef CONFIG_RCU_BOOST */
/*
static void rcu_preempt_check_blocked_tasks(struct rcu_node *rnp)
{
WARN_ON_ONCE(rcu_preempt_blocked_readers_cgp(rnp));
- if (!list_empty(&rnp->blkd_tasks))
+ if (rcu_preempt_has_tasks(rnp))
rnp->gp_tasks = rnp->blkd_tasks.next;
WARN_ON_ONCE(rnp->qsmask);
}
#ifdef CONFIG_HOTPLUG_CPU
-/*
- * Handle tasklist migration for case in which all CPUs covered by the
- * specified rcu_node have gone offline. Move them up to the root
- * rcu_node. The reason for not just moving them to the immediate
- * parent is to remove the need for rcu_read_unlock_special() to
- * make more than two attempts to acquire the target rcu_node's lock.
- * Returns true if there were tasks blocking the current RCU grace
- * period.
- *
- * Returns 1 if there was previously a task blocking the current grace
- * period on the specified rcu_node structure.
- *
- * The caller must hold rnp->lock with irqs disabled.
- */
-static int rcu_preempt_offline_tasks(struct rcu_state *rsp,
- struct rcu_node *rnp,
- struct rcu_data *rdp)
-{
- struct list_head *lp;
- struct list_head *lp_root;
- int retval = 0;
- struct rcu_node *rnp_root = rcu_get_root(rsp);
- struct task_struct *t;
-
- if (rnp == rnp_root) {
- WARN_ONCE(1, "Last CPU thought to be offlined?");
- return 0; /* Shouldn't happen: at least one CPU online. */
- }
-
- /* If we are on an internal node, complain bitterly. */
- WARN_ON_ONCE(rnp != rdp->mynode);
-
- /*
- * Move tasks up to root rcu_node. Don't try to get fancy for
- * this corner-case operation -- just put this node's tasks
- * at the head of the root node's list, and update the root node's
- * ->gp_tasks and ->exp_tasks pointers to those of this node's,
- * if non-NULL. This might result in waiting for more tasks than
- * absolutely necessary, but this is a good performance/complexity
- * tradeoff.
- */
- if (rcu_preempt_blocked_readers_cgp(rnp) && rnp->qsmask == 0)
- retval |= RCU_OFL_TASKS_NORM_GP;
- if (rcu_preempted_readers_exp(rnp))
- retval |= RCU_OFL_TASKS_EXP_GP;
- lp = &rnp->blkd_tasks;
- lp_root = &rnp_root->blkd_tasks;
- while (!list_empty(lp)) {
- t = list_entry(lp->next, typeof(*t), rcu_node_entry);
- raw_spin_lock(&rnp_root->lock); /* irqs already disabled */
- smp_mb__after_unlock_lock();
- list_del(&t->rcu_node_entry);
- t->rcu_blocked_node = rnp_root;
- list_add(&t->rcu_node_entry, lp_root);
- if (&t->rcu_node_entry == rnp->gp_tasks)
- rnp_root->gp_tasks = rnp->gp_tasks;
- if (&t->rcu_node_entry == rnp->exp_tasks)
- rnp_root->exp_tasks = rnp->exp_tasks;
-#ifdef CONFIG_RCU_BOOST
- if (&t->rcu_node_entry == rnp->boost_tasks)
- rnp_root->boost_tasks = rnp->boost_tasks;
-#endif /* #ifdef CONFIG_RCU_BOOST */
- raw_spin_unlock(&rnp_root->lock); /* irqs still disabled */
- }
-
- rnp->gp_tasks = NULL;
- rnp->exp_tasks = NULL;
-#ifdef CONFIG_RCU_BOOST
- rnp->boost_tasks = NULL;
- /*
- * In case root is being boosted and leaf was not. Make sure
- * that we boost the tasks blocking the current grace period
- * in this case.
- */
- raw_spin_lock(&rnp_root->lock); /* irqs already disabled */
- smp_mb__after_unlock_lock();
- if (rnp_root->boost_tasks != NULL &&
- rnp_root->boost_tasks != rnp_root->gp_tasks &&
- rnp_root->boost_tasks != rnp_root->exp_tasks)
- rnp_root->boost_tasks = rnp_root->gp_tasks;
- raw_spin_unlock(&rnp_root->lock); /* irqs still disabled */
-#endif /* #ifdef CONFIG_RCU_BOOST */
-
- return retval;
-}
-
#endif /* #ifdef CONFIG_HOTPLUG_CPU */
/*
raw_spin_lock_irqsave(&rnp->lock, flags);
smp_mb__after_unlock_lock();
- if (list_empty(&rnp->blkd_tasks)) {
+ if (!rcu_preempt_has_tasks(rnp)) {
raw_spin_unlock_irqrestore(&rnp->lock, flags);
} else {
rnp->exp_tasks = rnp->blkd_tasks.next;
rcu_bootup_announce_oddness();
}
-/*
- * Return the number of RCU batches processed thus far for debug & stats.
- */
-long rcu_batches_completed(void)
-{
- return rcu_batches_completed_sched();
-}
-EXPORT_SYMBOL_GPL(rcu_batches_completed);
-
/*
* Because preemptible RCU does not exist, we never have to check for
* CPUs being in quiescent states.
#ifdef CONFIG_HOTPLUG_CPU
-/* Because preemptible RCU does not exist, no quieting of tasks. */
-static void rcu_report_unblock_qs_rnp(struct rcu_node *rnp, unsigned long flags)
- __releases(rnp->lock)
+/*
+ * Because there is no preemptible RCU, there can be no readers blocked.
+ */
+static bool rcu_preempt_has_tasks(struct rcu_node *rnp)
{
- raw_spin_unlock_irqrestore(&rnp->lock, flags);
+ return false;
}
#endif /* #ifdef CONFIG_HOTPLUG_CPU */
WARN_ON_ONCE(rnp->qsmask);
}
-#ifdef CONFIG_HOTPLUG_CPU
-
-/*
- * Because preemptible RCU does not exist, it never needs to migrate
- * tasks that were blocked within RCU read-side critical sections, and
- * such non-existent tasks cannot possibly have been blocking the current
- * grace period.
- */
-static int rcu_preempt_offline_tasks(struct rcu_state *rsp,
- struct rcu_node *rnp,
- struct rcu_data *rdp)
-{
- return 0;
-}
-
-#endif /* #ifdef CONFIG_HOTPLUG_CPU */
-
/*
* Because preemptible RCU does not exist, it never has any callbacks
* to check.
}
EXPORT_SYMBOL_GPL(synchronize_rcu_expedited);
-#ifdef CONFIG_HOTPLUG_CPU
-
-/*
- * Because preemptible RCU does not exist, there is never any need to
- * report on tasks preempted in RCU read-side critical sections during
- * expedited RCU grace periods.
- */
-static void rcu_report_exp_rnp(struct rcu_state *rsp, struct rcu_node *rnp,
- bool wake)
-{
-}
-
-#endif /* #ifdef CONFIG_HOTPLUG_CPU */
-
/*
* Because preemptible RCU does not exist, rcu_barrier() is just
* another name for rcu_barrier_sched().
static void rcu_initiate_boost_trace(struct rcu_node *rnp)
{
- if (list_empty(&rnp->blkd_tasks))
+ if (!rcu_preempt_has_tasks(rnp))
rnp->n_balk_blkd_tasks++;
else if (rnp->exp_tasks == NULL && rnp->gp_tasks == NULL)
rnp->n_balk_exp_gp_tasks++;
struct task_struct *t;
struct list_head *tb;
- if (rnp->exp_tasks == NULL && rnp->boost_tasks == NULL)
+ if (ACCESS_ONCE(rnp->exp_tasks) == NULL &&
+ ACCESS_ONCE(rnp->boost_tasks) == NULL)
return 0; /* Nothing left to boost. */
raw_spin_lock_irqsave(&rnp->lock, flags);
*/
t = container_of(tb, struct task_struct, rcu_node_entry);
rt_mutex_init_proxy_locked(&rnp->boost_mtx, t);
- init_completion(&rnp->boost_completion);
raw_spin_unlock_irqrestore(&rnp->lock, flags);
/* Lock only for side effect: boosts task t's priority. */
rt_mutex_lock(&rnp->boost_mtx);
rt_mutex_unlock(&rnp->boost_mtx); /* Then keep lockdep happy. */
- /* Wait for boostee to be done w/boost_mtx before reinitializing. */
- wait_for_completion(&rnp->boost_completion);
-
return ACCESS_ONCE(rnp->exp_tasks) != NULL ||
ACCESS_ONCE(rnp->boost_tasks) != NULL;
}
for (cpu = rnp->grplo; cpu <= rnp->grphi; cpu++, mask >>= 1)
if ((mask & 0x1) && cpu != outgoingcpu)
cpumask_set_cpu(cpu, cm);
- if (cpumask_weight(cm) == 0) {
+ if (cpumask_weight(cm) == 0)
cpumask_setall(cm);
- for (cpu = rnp->grplo; cpu <= rnp->grphi; cpu++)
- cpumask_clear_cpu(cpu, cm);
- WARN_ON_ONCE(cpumask_weight(cm) == 0);
- }
set_cpus_allowed_ptr(t, cm);
free_cpumask_var(cm);
}
for_each_possible_cpu(cpu)
per_cpu(rcu_cpu_has_work, cpu) = 0;
BUG_ON(smpboot_register_percpu_thread(&rcu_cpu_thread_spec));
- rnp = rcu_get_root(rcu_state_p);
- (void)rcu_spawn_one_boost_kthread(rcu_state_p, rnp);
- if (NUM_RCU_NODES > 1) {
- rcu_for_each_leaf_node(rcu_state_p, rnp)
- (void)rcu_spawn_one_boost_kthread(rcu_state_p, rnp);
- }
+ rcu_for_each_leaf_node(rcu_state_p, rnp)
+ (void)rcu_spawn_one_boost_kthread(rcu_state_p, rnp);
}
static void rcu_prepare_kthreads(int cpu)
* completed since we last checked and there are
* callbacks not yet ready to invoke.
*/
- if (rdp->completed != rnp->completed &&
+ if ((rdp->completed != rnp->completed ||
+ unlikely(ACCESS_ONCE(rdp->gpwrap))) &&
rdp->nxttail[RCU_DONE_TAIL] != rdp->nxttail[RCU_NEXT_TAIL])
note_gp_changes(rsp, rdp);
ticks_value = rsp->gpnum - rdp->gpnum;
}
print_cpu_stall_fast_no_hz(fast_no_hz, cpu);
- pr_err("\t%d: (%lu %s) idle=%03x/%llx/%d softirq=%u/%u %s\n",
+ pr_err("\t%d: (%lu %s) idle=%03x/%llx/%d softirq=%u/%u fqs=%ld %s\n",
cpu, ticks_value, ticks_title,
atomic_read(&rdtp->dynticks) & 0xfff,
rdtp->dynticks_nesting, rdtp->dynticks_nmi_nesting,
rdp->softirq_snap, kstat_softirqs_cpu(RCU_SOFTIRQ, cpu),
+ ACCESS_ONCE(rsp->n_force_qs) - rsp->n_force_qs_gpstart,
fast_no_hz);
}
static bool rcu_nocb_cpu_needs_barrier(struct rcu_state *rsp, int cpu)
{
struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
+ unsigned long ret;
+#ifdef CONFIG_PROVE_RCU
struct rcu_head *rhp;
+#endif /* #ifdef CONFIG_PROVE_RCU */
- /* No-CBs CPUs might have callbacks on any of three lists. */
+ /*
+ * Check count of all no-CBs callbacks awaiting invocation.
+ * There needs to be a barrier before this function is called,
+ * but associated with a prior determination that no more
+ * callbacks would be posted. In the worst case, the first
+ * barrier in _rcu_barrier() suffices (but the caller cannot
+ * necessarily rely on this, not a substitute for the caller
+ * getting the concurrency design right!). There must also be
+ * a barrier between the following load an posting of a callback
+ * (if a callback is in fact needed). This is associated with an
+ * atomic_inc() in the caller.
+ */
+ ret = atomic_long_read(&rdp->nocb_q_count);
+
+#ifdef CONFIG_PROVE_RCU
rhp = ACCESS_ONCE(rdp->nocb_head);
if (!rhp)
rhp = ACCESS_ONCE(rdp->nocb_gp_head);
cpu, rhp->func);
WARN_ON_ONCE(1);
}
+#endif /* #ifdef CONFIG_PROVE_RCU */
- return !!rhp;
+ return !!ret;
}
/*
struct task_struct *t;
/* Enqueue the callback on the nocb list and update counts. */
+ atomic_long_add(rhcount, &rdp->nocb_q_count);
+ /* rcu_barrier() relies on ->nocb_q_count add before xchg. */
old_rhpp = xchg(&rdp->nocb_tail, rhtp);
ACCESS_ONCE(*old_rhpp) = rhp;
- atomic_long_add(rhcount, &rdp->nocb_q_count);
atomic_long_add(rhcount_lazy, &rdp->nocb_q_count_lazy);
smp_mb__after_atomic(); /* Store *old_rhpp before _wake test. */
/* Move callbacks to wait-for-GP list, which is empty. */
ACCESS_ONCE(rdp->nocb_head) = NULL;
rdp->nocb_gp_tail = xchg(&rdp->nocb_tail, &rdp->nocb_head);
- rdp->nocb_gp_count = atomic_long_xchg(&rdp->nocb_q_count, 0);
- rdp->nocb_gp_count_lazy =
- atomic_long_xchg(&rdp->nocb_q_count_lazy, 0);
gotcbs = true;
}
/* Append callbacks to follower's "done" list. */
tail = xchg(&rdp->nocb_follower_tail, rdp->nocb_gp_tail);
*tail = rdp->nocb_gp_head;
- atomic_long_add(rdp->nocb_gp_count, &rdp->nocb_follower_count);
- atomic_long_add(rdp->nocb_gp_count_lazy,
- &rdp->nocb_follower_count_lazy);
smp_mb__after_atomic(); /* Store *tail before wakeup. */
if (rdp != my_rdp && tail == &rdp->nocb_follower_head) {
/*
trace_rcu_nocb_wake(rdp->rsp->name, rdp->cpu, "WokeNonEmpty");
ACCESS_ONCE(rdp->nocb_follower_head) = NULL;
tail = xchg(&rdp->nocb_follower_tail, &rdp->nocb_follower_head);
- c = atomic_long_xchg(&rdp->nocb_follower_count, 0);
- cl = atomic_long_xchg(&rdp->nocb_follower_count_lazy, 0);
- rdp->nocb_p_count += c;
- rdp->nocb_p_count_lazy += cl;
/* Each pass through the following loop invokes a callback. */
- trace_rcu_batch_start(rdp->rsp->name, cl, c, -1);
+ trace_rcu_batch_start(rdp->rsp->name,
+ atomic_long_read(&rdp->nocb_q_count_lazy),
+ atomic_long_read(&rdp->nocb_q_count), -1);
c = cl = 0;
while (list) {
next = list->next;
list = next;
}
trace_rcu_batch_end(rdp->rsp->name, c, !!list, 0, 0, 1);
- ACCESS_ONCE(rdp->nocb_p_count) = rdp->nocb_p_count - c;
- ACCESS_ONCE(rdp->nocb_p_count_lazy) =
- rdp->nocb_p_count_lazy - cl;
+ smp_mb__before_atomic(); /* _add after CB invocation. */
+ atomic_long_add(-c, &rdp->nocb_q_count);
+ atomic_long_add(-cl, &rdp->nocb_q_count_lazy);
rdp->n_nocbs_invoked += c;
}
return 0;
#define RCU_TREE_NONCORE
#include "tree.h"
+DECLARE_PER_CPU_SHARED_ALIGNED(unsigned long, rcu_qs_ctr);
+
static int r_open(struct inode *inode, struct file *file,
const struct seq_operations *op)
{
if (!rdp->beenonline)
return;
- seq_printf(m, "%3d%cc=%ld g=%ld pq=%d qp=%d",
+ seq_printf(m, "%3d%cc=%ld g=%ld pq=%d/%d qp=%d",
rdp->cpu,
cpu_is_offline(rdp->cpu) ? '!' : ' ',
ulong2long(rdp->completed), ulong2long(rdp->gpnum),
- rdp->passed_quiesce, rdp->qs_pending);
+ rdp->passed_quiesce,
+ rdp->rcu_qs_ctr_snap == per_cpu(rcu_qs_ctr, rdp->cpu),
+ rdp->qs_pending);
seq_printf(m, " dt=%d/%llx/%d df=%lu",
atomic_read(&rdp->dynticks->dynticks),
rdp->dynticks->dynticks_nesting,
unsigned long flags;
int ret = 1;
+ /*
+ * Since x->done will need to be locked only
+ * in the non-blocking case, we check x->done
+ * first without taking the lock so we can
+ * return early in the blocking case.
+ */
+ if (!ACCESS_ONCE(x->done))
+ return 0;
+
spin_lock_irqsave(&x->wait.lock, flags);
if (!x->done)
ret = 0;
*/
bool completion_done(struct completion *x)
{
- unsigned long flags;
- int ret = 1;
-
- spin_lock_irqsave(&x->wait.lock, flags);
- if (!x->done)
- ret = 0;
- spin_unlock_irqrestore(&x->wait.lock, flags);
- return ret;
+ return !!ACCESS_ONCE(x->done);
}
EXPORT_SYMBOL(completion_done);
{
s64 delta;
- if (rq->skip_clock_update > 0)
+ lockdep_assert_held(&rq->lock);
+
+ if (rq->clock_skip_update & RQCF_ACT_SKIP)
return;
delta = sched_clock_cpu(cpu_of(rq)) - rq->clock;
*/
void hrtick_start(struct rq *rq, u64 delay)
{
+ /*
+ * Don't schedule slices shorter than 10000ns, that just
+ * doesn't make sense. Rely on vruntime for fairness.
+ */
+ delay = max_t(u64, delay, 10000LL);
__hrtimer_start_range_ns(&rq->hrtick_timer, ns_to_ktime(delay), 0,
HRTIMER_MODE_REL_PINNED, 0);
}
* this case, we can save a useless back to back clock update.
*/
if (task_on_rq_queued(rq->curr) && test_tsk_need_resched(rq->curr))
- rq->skip_clock_update = 1;
+ rq_clock_skip_update(rq, true);
}
#ifdef CONFIG_SMP
if (p->sched_class->migrate_task_rq)
p->sched_class->migrate_task_rq(p, new_cpu);
p->se.nr_migrations++;
- perf_sw_event(PERF_COUNT_SW_CPU_MIGRATIONS, 1, NULL, 0);
+ perf_sw_event_sched(PERF_COUNT_SW_CPU_MIGRATIONS, 1, 0);
}
__set_task_cpu(p, new_cpu);
dl_se->dl_period = 0;
dl_se->flags = 0;
dl_se->dl_bw = 0;
+
+ dl_se->dl_throttled = 0;
+ dl_se->dl_new = 1;
+ dl_se->dl_yielded = 0;
}
/*
p->se.prev_sum_exec_runtime = 0;
p->se.nr_migrations = 0;
p->se.vruntime = 0;
+#ifdef CONFIG_SMP
+ p->se.avg.decay_count = 0;
+#endif
INIT_LIST_HEAD(&p->se.group_node);
#ifdef CONFIG_SCHEDSTATS
#endif
RB_CLEAR_NODE(&p->dl.rb_node);
- hrtimer_init(&p->dl.dl_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+ init_dl_task_timer(&p->dl);
__dl_clear_params(p);
INIT_LIST_HEAD(&p->rt.run_list);
* allocated bandwidth to reflect the new situation.
*
* This function is called while holding p's rq->lock.
+ *
+ * XXX we should delay bw change until the task's 0-lag point, see
+ * __setparam_dl().
*/
static int dl_overflow(struct task_struct *p, int policy,
const struct sched_attr *attr)
* - explicit schedule() call
* - return from syscall or exception to user-space
* - return from interrupt-handler to user-space
+ *
+ * WARNING: all callers must re-check need_resched() afterward and reschedule
+ * accordingly in case an event triggered the need for rescheduling (such as
+ * an interrupt waking up a task) while preemption was disabled in __schedule().
*/
static void __sched __schedule(void)
{
struct rq *rq;
int cpu;
-need_resched:
preempt_disable();
cpu = smp_processor_id();
rq = cpu_rq(cpu);
smp_mb__before_spinlock();
raw_spin_lock_irq(&rq->lock);
+ rq->clock_skip_update <<= 1; /* promote REQ to ACT */
+
switch_count = &prev->nivcsw;
if (prev->state && !(preempt_count() & PREEMPT_ACTIVE)) {
if (unlikely(signal_pending_state(prev->state, prev))) {
switch_count = &prev->nvcsw;
}
- if (task_on_rq_queued(prev) || rq->skip_clock_update < 0)
+ if (task_on_rq_queued(prev))
update_rq_clock(rq);
next = pick_next_task(rq, prev);
clear_tsk_need_resched(prev);
clear_preempt_need_resched();
- rq->skip_clock_update = 0;
+ rq->clock_skip_update = 0;
if (likely(prev != next)) {
rq->nr_switches++;
post_schedule(rq);
sched_preempt_enable_no_resched();
- if (need_resched())
- goto need_resched;
}
static inline void sched_submit_work(struct task_struct *tsk)
struct task_struct *tsk = current;
sched_submit_work(tsk);
- __schedule();
+ do {
+ __schedule();
+ } while (need_resched());
}
EXPORT_SYMBOL(schedule);
preempt_disable();
}
+static void preempt_schedule_common(void)
+{
+ do {
+ __preempt_count_add(PREEMPT_ACTIVE);
+ __schedule();
+ __preempt_count_sub(PREEMPT_ACTIVE);
+
+ /*
+ * Check again in case we missed a preemption opportunity
+ * between schedule and now.
+ */
+ barrier();
+ } while (need_resched());
+}
+
#ifdef CONFIG_PREEMPT
/*
* this is the entry point to schedule() from in-kernel preemption
if (likely(!preemptible()))
return;
- do {
- __preempt_count_add(PREEMPT_ACTIVE);
- __schedule();
- __preempt_count_sub(PREEMPT_ACTIVE);
-
- /*
- * Check again in case we missed a preemption opportunity
- * between schedule and now.
- */
- barrier();
- } while (need_resched());
+ preempt_schedule_common();
}
NOKPROBE_SYMBOL(preempt_schedule);
EXPORT_SYMBOL(preempt_schedule);
{
struct sched_dl_entity *dl_se = &p->dl;
- init_dl_task_timer(dl_se);
dl_se->dl_runtime = attr->sched_runtime;
dl_se->dl_deadline = attr->sched_deadline;
dl_se->dl_period = attr->sched_period ?: dl_se->dl_deadline;
dl_se->flags = attr->sched_flags;
dl_se->dl_bw = to_ratio(dl_se->dl_period, dl_se->dl_runtime);
- dl_se->dl_throttled = 0;
- dl_se->dl_new = 1;
- dl_se->dl_yielded = 0;
+
+ /*
+ * Changing the parameters of a task is 'tricky' and we're not doing
+ * the correct thing -- also see task_dead_dl() and switched_from_dl().
+ *
+ * What we SHOULD do is delay the bandwidth release until the 0-lag
+ * point. This would include retaining the task_struct until that time
+ * and change dl_overflow() to not immediately decrement the current
+ * amount.
+ *
+ * Instead we retain the current runtime/deadline and let the new
+ * parameters take effect after the current reservation period lapses.
+ * This is safe (albeit pessimistic) because the 0-lag point is always
+ * before the current scheduling deadline.
+ *
+ * We can still have temporary overloads because we do not delay the
+ * change in bandwidth until that time; so admission control is
+ * not on the safe side. It does however guarantee tasks will never
+ * consume more than promised.
+ */
}
/*
return match;
}
+static bool dl_param_changed(struct task_struct *p,
+ const struct sched_attr *attr)
+{
+ struct sched_dl_entity *dl_se = &p->dl;
+
+ if (dl_se->dl_runtime != attr->sched_runtime ||
+ dl_se->dl_deadline != attr->sched_deadline ||
+ dl_se->dl_period != attr->sched_period ||
+ dl_se->flags != attr->sched_flags)
+ return true;
+
+ return false;
+}
+
static int __sched_setscheduler(struct task_struct *p,
const struct sched_attr *attr,
bool user)
goto change;
if (rt_policy(policy) && attr->sched_priority != p->rt_priority)
goto change;
- if (dl_policy(policy))
+ if (dl_policy(policy) && dl_param_changed(p, attr))
goto change;
p->sched_reset_on_fork = reset_on_fork;
return 0;
}
-static void __cond_resched(void)
-{
- __preempt_count_add(PREEMPT_ACTIVE);
- __schedule();
- __preempt_count_sub(PREEMPT_ACTIVE);
-}
-
int __sched _cond_resched(void)
{
if (should_resched()) {
- __cond_resched();
+ preempt_schedule_common();
return 1;
}
return 0;
if (spin_needbreak(lock) || resched) {
spin_unlock(lock);
if (resched)
- __cond_resched();
+ preempt_schedule_common();
else
cpu_relax();
ret = 1;
if (should_resched()) {
local_bh_enable();
- __cond_resched();
+ preempt_schedule_common();
local_bh_disable();
return 1;
}
{
unsigned long free = 0;
int ppid;
- unsigned state;
+ unsigned long state = p->state;
- state = p->state ? __ffs(p->state) + 1 : 0;
+ if (state)
+ state = __ffs(state) + 1;
printk(KERN_INFO "%-15.15s %c", p->comm,
state < sizeof(stat_nam) - 1 ? stat_nam[state] : '?');
#if BITS_PER_LONG == 32
struct dl_bw *cur_dl_b;
unsigned long flags;
+ if (!cpumask_weight(cur))
+ return ret;
+
rcu_read_lock_sched();
cur_dl_b = dl_bw_of(cpumask_any(cur));
trial_cpus = cpumask_weight(trial);
void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask)
{
- if (p->sched_class && p->sched_class->set_cpus_allowed)
+ if (p->sched_class->set_cpus_allowed)
p->sched_class->set_cpus_allowed(p, new_mask);
cpumask_copy(&p->cpus_allowed, new_mask);
atomic_inc(&init_mm.mm_count);
enter_lazy_tlb(&init_mm, current);
+ /*
+ * During early bootup we pretend to be a normal task:
+ */
+ current->sched_class = &fair_sched_class;
+
/*
* Make us the idle thread. Technically, schedule() should not be
* called from this thread, however somewhere below it might be,
calc_load_update = jiffies + LOAD_FREQ;
- /*
- * During early bootup we pretend to be a normal task:
- */
- current->sched_class = &fair_sched_class;
-
#ifdef CONFIG_SMP
zalloc_cpumask_var(&sched_domains_tmpmask, GFP_NOWAIT);
/* May be allocated at isolcpus cmdline parse time */
* since we will exit with TASK_RUNNING make sure we enter with it,
* otherwise we will destroy state.
*/
- if (WARN_ONCE(current->state != TASK_RUNNING,
+ WARN_ONCE(current->state != TASK_RUNNING && current->task_state_change,
"do not call blocking ops when !TASK_RUNNING; "
"state=%lx set at [<%p>] %pS\n",
current->state,
(void *)current->task_state_change,
- (void *)current->task_state_change))
- __set_current_state(TASK_RUNNING);
+ (void *)current->task_state_change);
___might_sleep(file, line, preempt_offset);
}
in_atomic(), irqs_disabled(),
current->pid, current->comm);
+ if (task_stack_end_corrupted(current))
+ printk(KERN_EMERG "Thread overran stack, or stack corrupted\n");
+
debug_show_held_locks(current);
if (irqs_disabled())
print_irqtrace_events(current);
int best_cpu = -1;
const struct sched_dl_entity *dl_se = &p->dl;
- if (later_mask && cpumask_and(later_mask, later_mask, cp->free_cpus)) {
+ if (later_mask &&
+ cpumask_and(later_mask, cp->free_cpus, &p->cpus_allowed)) {
best_cpu = cpumask_any(later_mask);
goto out;
} else if (cpumask_test_cpu(cpudl_maximum(cp), &p->cpus_allowed) &&
raw_spin_unlock_irqrestore(&cp->lock, flags);
}
+/*
+ * cpudl_set_freecpu - Set the cpudl.free_cpus
+ * @cp: the cpudl max-heap context
+ * @cpu: rd attached cpu
+ */
+void cpudl_set_freecpu(struct cpudl *cp, int cpu)
+{
+ cpumask_set_cpu(cpu, cp->free_cpus);
+}
+
+/*
+ * cpudl_clear_freecpu - Clear the cpudl.free_cpus
+ * @cp: the cpudl max-heap context
+ * @cpu: rd attached cpu
+ */
+void cpudl_clear_freecpu(struct cpudl *cp, int cpu)
+{
+ cpumask_clear_cpu(cpu, cp->free_cpus);
+}
+
/*
* cpudl_init - initialize the cpudl structure
* @cp: the cpudl max-heap context
if (!cp->elements)
return -ENOMEM;
- if (!alloc_cpumask_var(&cp->free_cpus, GFP_KERNEL)) {
+ if (!zalloc_cpumask_var(&cp->free_cpus, GFP_KERNEL)) {
kfree(cp->elements);
return -ENOMEM;
}
for_each_possible_cpu(i)
cp->elements[i].idx = IDX_INVALID;
- cpumask_setall(cp->free_cpus);
-
return 0;
}
struct cpumask *later_mask);
void cpudl_set(struct cpudl *cp, int cpu, u64 dl, int is_valid);
int cpudl_init(struct cpudl *cp);
+void cpudl_set_freecpu(struct cpudl *cp, int cpu);
+void cpudl_clear_freecpu(struct cpudl *cp, int cpu);
void cpudl_cleanup(struct cpudl *cp);
#endif /* CONFIG_SMP */
dl_se->deadline = rq_clock(rq) + pi_se->dl_deadline;
dl_se->runtime = pi_se->dl_runtime;
}
+
+ if (dl_se->dl_yielded)
+ dl_se->dl_yielded = 0;
+ if (dl_se->dl_throttled)
+ dl_se->dl_throttled = 0;
}
/*
sched_clock_tick();
update_rq_clock(rq);
- dl_se->dl_throttled = 0;
- dl_se->dl_yielded = 0;
- if (task_on_rq_queued(p)) {
- enqueue_task_dl(rq, p, ENQUEUE_REPLENISH);
- if (dl_task(rq->curr))
- check_preempt_curr_dl(rq, p, 0);
- else
- resched_curr(rq);
+ enqueue_task_dl(rq, p, ENQUEUE_REPLENISH);
+ if (dl_task(rq->curr))
+ check_preempt_curr_dl(rq, p, 0);
+ else
+ resched_curr(rq);
#ifdef CONFIG_SMP
- /*
- * Queueing this task back might have overloaded rq,
- * check if we need to kick someone away.
- */
- if (has_pushable_dl_tasks(rq))
- push_dl_task(rq);
+ /*
+ * Queueing this task back might have overloaded rq,
+ * check if we need to kick someone away.
+ */
+ if (has_pushable_dl_tasks(rq))
+ push_dl_task(rq);
#endif
- }
unlock:
raw_spin_unlock(&rq->lock);
dl_se->runtime -= dl_se->dl_yielded ? 0 : delta_exec;
if (dl_runtime_exceeded(rq, dl_se)) {
+ dl_se->dl_throttled = 1;
__dequeue_task_dl(rq, curr, 0);
- if (likely(start_dl_timer(dl_se, curr->dl.dl_boosted)))
- dl_se->dl_throttled = 1;
- else
+ if (unlikely(!start_dl_timer(dl_se, curr->dl.dl_boosted)))
enqueue_task_dl(rq, curr, ENQUEUE_REPLENISH);
if (!is_leftmost(curr, &rq->dl))
* its rq, the bandwidth timer callback (which clearly has not
* run yet) will take care of this.
*/
- if (p->dl.dl_throttled)
+ if (p->dl.dl_throttled && !(flags & ENQUEUE_REPLENISH))
return;
enqueue_dl_entity(&p->dl, pi_se, flags);
{
update_curr_dl(rq);
- if (hrtick_enabled(rq) && queued && p->dl.runtime > 0)
+ /*
+ * Even when we have runtime, update_curr_dl() might have resulted in us
+ * not being the leftmost task anymore. In that case NEED_RESCHED will
+ * be set and schedule() will start a new hrtick for the next task.
+ */
+ if (hrtick_enabled(rq) && queued && p->dl.runtime > 0 &&
+ is_leftmost(p, &rq->dl))
start_hrtick_dl(rq, p);
}
* Since we are TASK_DEAD we won't slip out of the domain!
*/
raw_spin_lock_irq(&dl_b->lock);
+ /* XXX we should retain the bw until 0-lag */
dl_b->total_bw -= p->dl.dl_bw;
raw_spin_unlock_irq(&dl_b->lock);
* We have to consider system topology and task affinity
* first, then we can look for a suitable cpu.
*/
- cpumask_copy(later_mask, task_rq(task)->rd->span);
- cpumask_and(later_mask, later_mask, cpu_active_mask);
- cpumask_and(later_mask, later_mask, &task->cpus_allowed);
best_cpu = cpudl_find(&task_rq(task)->rd->cpudl,
task, later_mask);
if (best_cpu == -1)
if (rq->dl.overloaded)
dl_set_overload(rq);
+ cpudl_set_freecpu(&rq->rd->cpudl, rq->cpu);
if (rq->dl.dl_nr_running > 0)
cpudl_set(&rq->rd->cpudl, rq->cpu, rq->dl.earliest_dl.curr, 1);
}
dl_clear_overload(rq);
cpudl_set(&rq->rd->cpudl, rq->cpu, 0, 0);
+ cpudl_clear_freecpu(&rq->rd->cpudl, rq->cpu);
}
void init_sched_dl_class(void)
static void switched_from_dl(struct rq *rq, struct task_struct *p)
{
+ /* XXX we should retain the bw until 0-lag */
cancel_dl_timer(rq, p);
-
__dl_clear_params(p);
/*
PN(next_balance);
SEQ_printf(m, " .%-30s: %ld\n", "curr->pid", (long)(task_pid_nr(rq->curr)));
PN(clock);
+ PN(clock_task);
P(cpu_load[0]);
P(cpu_load[1]);
P(cpu_load[2]);
{
u32 slice;
- p->se.avg.decay_count = 0;
slice = sched_slice(task_cfs_rq(p), &p->se) >> 10;
p->se.avg.runnable_avg_sum = slice;
p->se.avg.runnable_avg_period = slice;
nodes = node_online_map;
for (dist = sched_max_numa_distance; dist > LOCAL_DISTANCE; dist--) {
unsigned long max_faults = 0;
- nodemask_t max_group;
+ nodemask_t max_group = NODE_MASK_NONE;
int a, b;
/* Are there nodes at this distance from each other? */
u64 decays = atomic64_read(&cfs_rq->decay_counter);
decays -= se->avg.decay_count;
+ se->avg.decay_count = 0;
if (!decays)
return 0;
se->avg.load_avg_contrib = decay_load(se->avg.load_avg_contrib, decays);
- se->avg.decay_count = 0;
return decays;
}
* so we don't do microscopic update in schedule()
* and double the fastpath cost.
*/
- rq->skip_clock_update = 1;
+ rq_clock_skip_update(rq, true);
}
set_skip_buddy(se);
*/
age_stamp = ACCESS_ONCE(rq->age_stamp);
avg = ACCESS_ONCE(rq->rt_avg);
+ delta = __rq_clock_broken(rq) - age_stamp;
- delta = rq_clock(rq) - age_stamp;
if (unlikely(delta < 0))
delta = 0;
rcu_idle_enter();
trace_cpu_idle_rcuidle(0, smp_processor_id());
local_irq_enable();
- while (!tif_need_resched())
+ while (!tif_need_resched() &&
+ (cpu_idle_force_poll || tick_check_broadcast_expired()))
cpu_relax();
trace_cpu_idle_rcuidle(PWR_EVENT_EXIT, smp_processor_id());
rcu_idle_exit();
enqueue = 1;
/*
- * Force a clock update if the CPU was idle,
- * lest wakeup -> unthrottle time accumulate.
+ * When we're idle and a woken (rt) task is
+ * throttled check_preempt_curr() will set
+ * skip_update and the time between the wakeup
+ * and this unthrottle will get accounted as
+ * 'runtime'.
*/
if (rt_rq->rt_nr_running && rq->curr == rq->idle)
- rq->skip_clock_update = -1;
+ rq_clock_skip_update(rq, false);
}
if (rt_rq->rt_time || rt_rq->rt_nr_running)
idle = 0;
curr->prio <= p->prio)) {
int target = find_lowest_rq(p);
- if (target != -1)
+ /*
+ * Don't bother moving it if the destination CPU is
+ * not running a lower priority task.
+ */
+ if (target != -1 &&
+ p->prio < cpu_rq(target)->rt.highest_prio.curr)
cpu = target;
}
rcu_read_unlock();
lowest_rq = cpu_rq(cpu);
+ if (lowest_rq->rt.highest_prio.curr <= task->prio) {
+ /*
+ * Target rq has tasks of equal or higher priority,
+ * retrying does not release any lock and is unlikely
+ * to yield a different result.
+ */
+ lowest_rq = NULL;
+ break;
+ }
+
/* if the prio of this runqueue changed, try again */
if (double_lock_balance(rq, lowest_rq)) {
/*
#ifdef CONFIG_NO_HZ_FULL
unsigned long last_sched_tick;
#endif
- int skip_clock_update;
-
/* capture load from *all* tasks on this cpu: */
struct load_weight load;
unsigned long nr_load_updates;
unsigned long next_balance;
struct mm_struct *prev_mm;
+ unsigned int clock_skip_update;
u64 clock;
u64 clock_task;
#define cpu_curr(cpu) (cpu_rq(cpu)->curr)
#define raw_rq() raw_cpu_ptr(&runqueues)
+static inline u64 __rq_clock_broken(struct rq *rq)
+{
+ return ACCESS_ONCE(rq->clock);
+}
+
static inline u64 rq_clock(struct rq *rq)
{
+ lockdep_assert_held(&rq->lock);
return rq->clock;
}
static inline u64 rq_clock_task(struct rq *rq)
{
+ lockdep_assert_held(&rq->lock);
return rq->clock_task;
}
+#define RQCF_REQ_SKIP 0x01
+#define RQCF_ACT_SKIP 0x02
+
+static inline void rq_clock_skip_update(struct rq *rq, bool skip)
+{
+ lockdep_assert_held(&rq->lock);
+ if (skip)
+ rq->clock_skip_update |= RQCF_REQ_SKIP;
+ else
+ rq->clock_skip_update &= ~RQCF_REQ_SKIP;
+}
+
#ifdef CONFIG_NUMA
enum numa_topology_type {
NUMA_DIRECT,
unsigned int cpu;
int ret = 0;
+ get_online_cpus();
mutex_lock(&smpboot_threads_lock);
for_each_online_cpu(cpu) {
ret = __smpboot_create_thread(plug_thread, cpu);
list_add(&plug_thread->list, &hotplug_threads);
out:
mutex_unlock(&smpboot_threads_lock);
+ put_online_cpus();
return ret;
}
EXPORT_SYMBOL_GPL(smpboot_register_percpu_thread);
trace_softirqs_off(ip);
raw_local_irq_restore(flags);
- if (preempt_count() == cnt)
+ if (preempt_count() == cnt) {
+#ifdef CONFIG_DEBUG_PREEMPT
+ current->preempt_disable_ip = get_parent_ip(CALLER_ADDR1);
+#endif
trace_preempt_off(CALLER_ADDR0, get_parent_ip(CALLER_ADDR1));
+ }
}
EXPORT_SYMBOL(__local_bh_disable_ip);
#endif /* CONFIG_TRACE_IRQFLAGS */
* in the task stack here.
*/
__do_softirq();
- rcu_note_context_switch();
local_irq_enable();
- cond_resched();
+ cond_resched_rcu_qs();
return;
}
local_irq_enable();
mono = ktime_get_update_offsets_tick(&off_real, &off_boot, &off_tai);
boot = ktime_add(mono, off_boot);
xtim = ktime_add(mono, off_real);
- tai = ktime_add(xtim, off_tai);
+ tai = ktime_add(mono, off_tai);
base->clock_base[HRTIMER_BASE_REALTIME].softirq_time = xtim;
base->clock_base[HRTIMER_BASE_MONOTONIC].softirq_time = mono;
/*
* Divide a ktime value by a nanosecond value
*/
-u64 ktime_divns(const ktime_t kt, s64 div)
+u64 __ktime_divns(const ktime_t kt, s64 div)
{
u64 dclc;
int sft = 0;
return dclc;
}
-EXPORT_SYMBOL_GPL(ktime_divns);
+EXPORT_SYMBOL_GPL(__ktime_divns);
#endif /* BITS_PER_LONG >= 64 */
/*
trace_hrtimer_cancel(timer);
}
+#if defined(CONFIG_NO_HZ_COMMON) || defined(CONFIG_HIGH_RES_TIMERS)
+static ktime_t __hrtimer_get_next_event(struct hrtimer_cpu_base *cpu_base)
+{
+ struct hrtimer_clock_base *base = cpu_base->clock_base;
+ ktime_t expires, expires_next = { .tv64 = KTIME_MAX };
+ int i;
+
+ for (i = 0; i < HRTIMER_MAX_CLOCK_BASES; i++, base++) {
+ struct timerqueue_node *next;
+ struct hrtimer *timer;
+
+ next = timerqueue_getnext(&base->active);
+ if (!next)
+ continue;
+
+ timer = container_of(next, struct hrtimer, node);
+ expires = ktime_sub(hrtimer_get_expires(timer), base->offset);
+ if (expires.tv64 < expires_next.tv64)
+ expires_next = expires;
+ }
+ /*
+ * clock_was_set() might have changed base->offset of any of
+ * the clock bases so the result might be negative. Fix it up
+ * to prevent a false positive in clockevents_program_event().
+ */
+ if (expires_next.tv64 < 0)
+ expires_next.tv64 = 0;
+ return expires_next;
+}
+#endif
+
/* High resolution timer related functions */
#ifdef CONFIG_HIGH_RES_TIMERS
static void
hrtimer_force_reprogram(struct hrtimer_cpu_base *cpu_base, int skip_equal)
{
- int i;
- struct hrtimer_clock_base *base = cpu_base->clock_base;
- ktime_t expires, expires_next;
-
- expires_next.tv64 = KTIME_MAX;
-
- for (i = 0; i < HRTIMER_MAX_CLOCK_BASES; i++, base++) {
- struct hrtimer *timer;
- struct timerqueue_node *next;
-
- next = timerqueue_getnext(&base->active);
- if (!next)
- continue;
- timer = container_of(next, struct hrtimer, node);
-
- expires = ktime_sub(hrtimer_get_expires(timer), base->offset);
- /*
- * clock_was_set() has changed base->offset so the
- * result might be negative. Fix it up to prevent a
- * false positive in clockevents_program_event()
- */
- if (expires.tv64 < 0)
- expires.tv64 = 0;
- if (expires.tv64 < expires_next.tv64)
- expires_next = expires;
- }
+ ktime_t expires_next = __hrtimer_get_next_event(cpu_base);
if (skip_equal && expires_next.tv64 == cpu_base->expires_next.tv64)
return;
if (expires.tv64 >= cpu_base->expires_next.tv64)
return 0;
+ /*
+ * When the target cpu of the timer is currently executing
+ * hrtimer_interrupt(), then we do not touch the clock event
+ * device. hrtimer_interrupt() will reevaluate all clock bases
+ * before reprogramming the device.
+ */
+ if (cpu_base->in_hrtirq)
+ return 0;
+
/*
* If a hang was detected in the last timer interrupt then we
* do not schedule a timer which is earlier than the expiry
ktime_t hrtimer_get_next_event(void)
{
struct hrtimer_cpu_base *cpu_base = this_cpu_ptr(&hrtimer_bases);
- struct hrtimer_clock_base *base = cpu_base->clock_base;
- ktime_t delta, mindelta = { .tv64 = KTIME_MAX };
+ ktime_t mindelta = { .tv64 = KTIME_MAX };
unsigned long flags;
- int i;
raw_spin_lock_irqsave(&cpu_base->lock, flags);
- if (!hrtimer_hres_active()) {
- for (i = 0; i < HRTIMER_MAX_CLOCK_BASES; i++, base++) {
- struct hrtimer *timer;
- struct timerqueue_node *next;
-
- next = timerqueue_getnext(&base->active);
- if (!next)
- continue;
-
- timer = container_of(next, struct hrtimer, node);
- delta.tv64 = hrtimer_get_expires_tv64(timer);
- delta = ktime_sub(delta, base->get_time());
- if (delta.tv64 < mindelta.tv64)
- mindelta.tv64 = delta.tv64;
- }
- }
+ if (!hrtimer_hres_active())
+ mindelta = ktime_sub(__hrtimer_get_next_event(cpu_base),
+ ktime_get());
raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
raw_spin_lock(&cpu_base->lock);
entry_time = now = hrtimer_update_base(cpu_base);
retry:
- expires_next.tv64 = KTIME_MAX;
+ cpu_base->in_hrtirq = 1;
/*
* We set expires_next to KTIME_MAX here with cpu_base->lock
* held to prevent that a timer is enqueued in our queue via
* are right-of a not yet expired timer, because that
* timer will have to trigger a wakeup anyway.
*/
-
- if (basenow.tv64 < hrtimer_get_softexpires_tv64(timer)) {
- ktime_t expires;
-
- expires = ktime_sub(hrtimer_get_expires(timer),
- base->offset);
- if (expires.tv64 < 0)
- expires.tv64 = KTIME_MAX;
- if (expires.tv64 < expires_next.tv64)
- expires_next = expires;
+ if (basenow.tv64 < hrtimer_get_softexpires_tv64(timer))
break;
- }
__run_hrtimer(timer, &basenow);
}
}
-
+ /* Reevaluate the clock bases for the next expiry */
+ expires_next = __hrtimer_get_next_event(cpu_base);
/*
* Store the new expiry value so the migration code can verify
* against it.
*/
cpu_base->expires_next = expires_next;
+ cpu_base->in_hrtirq = 0;
raw_spin_unlock(&cpu_base->lock);
/* Reprogramming necessary ? */
getnstimeofday64(&now);
if (abs(now.tv_nsec - (NSEC_PER_SEC / 2)) <= tick_nsec * 5) {
- struct timespec adjust = timespec64_to_timespec(now);
+ struct timespec64 adjust = now;
fail = -ENODEV;
if (persistent_clock_is_local)
adjust.tv_sec -= (sys_tz.tz_minuteswest * 60);
#ifdef CONFIG_GENERIC_CMOS_UPDATE
- fail = update_persistent_clock(adjust);
+ fail = update_persistent_clock(timespec64_to_timespec(adjust));
#endif
#ifdef CONFIG_RTC_SYSTOHC
if (fail == -ENODEV)
}
/**
- * getboottime - Return the real time of system boot.
- * @ts: pointer to the timespec to be set
+ * getboottime64 - Return the real time of system boot.
+ * @ts: pointer to the timespec64 to be set
*
- * Returns the wall-time of boot in a timespec.
+ * Returns the wall-time of boot in a timespec64.
*
* This is based on the wall_to_monotonic offset and the total suspend
* time. Calls to settimeofday will affect the value returned (which
* basically means that however wrong your real time clock is at boot time,
* you get the right time here).
*/
-void getboottime(struct timespec *ts)
+void getboottime64(struct timespec64 *ts)
{
struct timekeeper *tk = &tk_core.timekeeper;
ktime_t t = ktime_sub(tk->offs_real, tk->offs_boot);
- *ts = ktime_to_timespec(t);
+ *ts = ktime_to_timespec64(t);
}
-EXPORT_SYMBOL_GPL(getboottime);
+EXPORT_SYMBOL_GPL(getboottime64);
unsigned long get_seconds(void)
{
}
void *perf_trace_buf_prepare(int size, unsigned short type,
- struct pt_regs *regs, int *rctxp)
+ struct pt_regs **regs, int *rctxp)
{
struct trace_entry *entry;
unsigned long flags;
if (*rctxp < 0)
return NULL;
+ if (regs)
+ *regs = this_cpu_ptr(&__perf_regs[*rctxp]);
raw_data = this_cpu_ptr(perf_trace_buf[*rctxp]);
/* zero the dead bytes from align to not leak stack to user */
size = ALIGN(__size + sizeof(u32), sizeof(u64));
size -= sizeof(u32);
- entry = perf_trace_buf_prepare(size, call->event.type, regs, &rctx);
+ entry = perf_trace_buf_prepare(size, call->event.type, NULL, &rctx);
if (!entry)
return;
size = ALIGN(__size + sizeof(u32), sizeof(u64));
size -= sizeof(u32);
- entry = perf_trace_buf_prepare(size, call->event.type, regs, &rctx);
+ entry = perf_trace_buf_prepare(size, call->event.type, NULL, &rctx);
if (!entry)
return;
size -= sizeof(u32);
rec = (struct syscall_trace_enter *)perf_trace_buf_prepare(size,
- sys_data->enter_event->event.type, regs, &rctx);
+ sys_data->enter_event->event.type, NULL, &rctx);
if (!rec)
return;
size -= sizeof(u32);
rec = (struct syscall_trace_exit *)perf_trace_buf_prepare(size,
- sys_data->exit_event->event.type, regs, &rctx);
+ sys_data->exit_event->event.type, NULL, &rctx);
if (!rec)
return;
if (hlist_empty(head))
goto out;
- entry = perf_trace_buf_prepare(size, call->event.type, regs, &rctx);
+ entry = perf_trace_buf_prepare(size, call->event.type, NULL, &rctx);
if (!entry)
goto out;
tristate "torture tests for RCU"
depends on DEBUG_KERNEL
select TORTURE_TEST
+ select SRCU
default n
help
This option provides a kernel module that runs torture tests
config RCU_CPU_STALL_INFO
bool "Print additional diagnostics on RCU CPU stall"
depends on (TREE_RCU || PREEMPT_RCU) && DEBUG_KERNEL
- default n
+ default y
help
For each stalled CPU that is aware of the current RCU grace
period, print out additional per-CPU diagnostic information
EXPORT_SYMBOL(csum_partial_copy);
#ifndef csum_tcpudp_nofold
+static inline u32 from64to32(u64 x)
+{
+ /* add up 32-bit and 32-bit for 32+c bit */
+ x = (x & 0xffffffff) + (x >> 32);
+ /* add up carry.. */
+ x = (x & 0xffffffff) + (x >> 32);
+ return (u32)x;
+}
+
__wsum csum_tcpudp_nofold(__be32 saddr, __be32 daddr,
unsigned short len,
unsigned short proto,
#else
s += (proto + len) << 8;
#endif
- s += (s >> 32);
- return (__force __wsum)s;
+ return (__force __wsum)from64to32(s);
}
EXPORT_SYMBOL(csum_tcpudp_nofold);
#endif
config MMU_NOTIFIER
bool
+ select SRCU
config KSM
bool "Enable KSM for page merging"
return -ENOMEM;
if (ret & (VM_FAULT_HWPOISON | VM_FAULT_HWPOISON_LARGE))
return *flags & FOLL_HWPOISON ? -EHWPOISON : -EFAULT;
- if (ret & VM_FAULT_SIGBUS)
+ if (ret & (VM_FAULT_SIGBUS | VM_FAULT_SIGSEGV))
return -EFAULT;
BUG();
}
return -ENOMEM;
if (ret & (VM_FAULT_HWPOISON | VM_FAULT_HWPOISON_LARGE))
return -EHWPOISON;
- if (ret & VM_FAULT_SIGBUS)
+ if (ret & (VM_FAULT_SIGBUS | VM_FAULT_SIGSEGV))
return -EFAULT;
BUG();
}
else
ret = VM_FAULT_WRITE;
put_page(page);
- } while (!(ret & (VM_FAULT_WRITE | VM_FAULT_SIGBUS | VM_FAULT_OOM)));
+ } while (!(ret & (VM_FAULT_WRITE | VM_FAULT_SIGBUS | VM_FAULT_SIGSEGV | VM_FAULT_OOM)));
/*
* We must loop because handle_mm_fault() may back out if there's
* any difficulty e.g. if pte accessed bit gets updated concurrently.
* mem_cgroup_migrate - migrate a charge to another page
* @oldpage: currently charged page
* @newpage: page to transfer the charge to
- * @lrucare: both pages might be on the LRU already
+ * @lrucare: either or both pages might be on the LRU already
*
* Migrate the charge from @oldpage to @newpage.
*
/* Check if we need to add a guard page to the stack */
if (check_stack_guard_page(vma, address) < 0)
- return VM_FAULT_SIGBUS;
+ return VM_FAULT_SIGSEGV;
/* Use the zero-page for reads */
if (!(flags & FAULT_FLAG_WRITE) && !mm_forbids_zeropage(mm)) {
#endif
void *high_memory;
+EXPORT_SYMBOL(high_memory);
struct page *mem_map;
unsigned long max_mapnr;
unsigned long highest_memmap_pfn;
*/
if ((vma->vm_start <= addr) &&
(vma->vm_flags & VM_PFNMAP)) {
- next = vma->vm_end;
+ if (walk->pte_hole)
+ err = walk->pte_hole(addr, next, walk);
+ if (err)
+ break;
pgd = pgd_offset(walk->mm, next);
continue;
}
*/
oldpage = newpage;
} else {
- mem_cgroup_migrate(oldpage, newpage, false);
+ mem_cgroup_migrate(oldpage, newpage, true);
lru_cache_add_anon(newpage);
*pagep = newpage;
}
data[NFT_REG_VERDICT].verdict = NF_DROP;
}
-static int nft_reject_bridge_validate_hooks(const struct nft_chain *chain)
+static int nft_reject_bridge_validate(const struct nft_ctx *ctx,
+ const struct nft_expr *expr,
+ const struct nft_data **data)
{
- struct nft_base_chain *basechain;
-
- if (chain->flags & NFT_BASE_CHAIN) {
- basechain = nft_base_chain(chain);
-
- switch (basechain->ops[0].hooknum) {
- case NF_BR_PRE_ROUTING:
- case NF_BR_LOCAL_IN:
- break;
- default:
- return -EOPNOTSUPP;
- }
- }
- return 0;
+ return nft_chain_validate_hooks(ctx->chain, (1 << NF_BR_PRE_ROUTING) |
+ (1 << NF_BR_LOCAL_IN));
}
static int nft_reject_bridge_init(const struct nft_ctx *ctx,
struct nft_reject *priv = nft_expr_priv(expr);
int icmp_code, err;
- err = nft_reject_bridge_validate_hooks(ctx->chain);
+ err = nft_reject_bridge_validate(ctx, expr, NULL);
if (err < 0)
return err;
return -1;
}
-static int nft_reject_bridge_validate(const struct nft_ctx *ctx,
- const struct nft_expr *expr,
- const struct nft_data **data)
-{
- return nft_reject_bridge_validate_hooks(ctx->chain);
-}
-
static struct nft_expr_type nft_reject_bridge_type;
static const struct nft_expr_ops nft_reject_bridge_ops = {
.type = &nft_reject_bridge_type,
ASSERT_RTNL();
caifdev = netdev_priv(dev);
caif_netlink_parms(data, &caifdev->conn_req);
- dev_net_set(caifdev->netdev, src_net);
ret = register_netdevice(dev);
if (ret)
__be16 skb_network_protocol(struct sk_buff *skb, int *depth)
{
- unsigned int vlan_depth = skb->mac_len;
__be16 type = skb->protocol;
/* Tunnel gso handlers can set protocol to ethernet. */
type = eth->h_proto;
}
- /* if skb->protocol is 802.1Q/AD then the header should already be
- * present at mac_len - VLAN_HLEN (if mac_len > 0), or at
- * ETH_HLEN otherwise
- */
- if (type == htons(ETH_P_8021Q) || type == htons(ETH_P_8021AD)) {
- if (vlan_depth) {
- if (WARN_ON(vlan_depth < VLAN_HLEN))
- return 0;
- vlan_depth -= VLAN_HLEN;
- } else {
- vlan_depth = ETH_HLEN;
- }
- do {
- struct vlan_hdr *vh;
-
- if (unlikely(!pskb_may_pull(skb,
- vlan_depth + VLAN_HLEN)))
- return 0;
-
- vh = (struct vlan_hdr *)(skb->data + vlan_depth);
- type = vh->h_vlan_encapsulated_proto;
- vlan_depth += VLAN_HLEN;
- } while (type == htons(ETH_P_8021Q) ||
- type == htons(ETH_P_8021AD));
- }
-
- *depth = vlan_depth;
-
- return type;
+ return __vlan_get_protocol(skb, type, depth);
}
/**
}
EXPORT_SYMBOL(netdev_upper_dev_unlink);
-void netdev_adjacent_add_links(struct net_device *dev)
+static void netdev_adjacent_add_links(struct net_device *dev)
{
struct netdev_adjacent *iter;
}
}
-void netdev_adjacent_del_links(struct net_device *dev)
+static void netdev_adjacent_del_links(struct net_device *dev)
{
struct netdev_adjacent *iter;
if (!queue)
return NULL;
netdev_init_one_queue(dev, queue, NULL);
- queue->qdisc = &noop_qdisc;
+ RCU_INIT_POINTER(queue->qdisc, &noop_qdisc);
queue->qdisc_sleeping = &noop_qdisc;
rcu_assign_pointer(dev->ingress_queue, queue);
#endif
goto errout;
}
+ if (!skb->len)
+ goto errout;
+
rtnl_notify(skb, net, 0, RTNLGRP_LINK, NULL, GFP_ATOMIC);
return 0;
errout:
WARN_ON(err == -EMSGSIZE);
kfree_skb(skb);
- rtnl_set_sk_err(net, RTNLGRP_LINK, err);
+ if (err)
+ rtnl_set_sk_err(net, RTNLGRP_LINK, err);
return err;
}
/*
* Generic function to send a packet as reply to another packet.
* Used to send some TCP resets/acks so far.
- *
- * Use a fake percpu inet socket to avoid false sharing and contention.
*/
-static DEFINE_PER_CPU(struct inet_sock, unicast_sock) = {
- .sk = {
- .__sk_common = {
- .skc_refcnt = ATOMIC_INIT(1),
- },
- .sk_wmem_alloc = ATOMIC_INIT(1),
- .sk_allocation = GFP_ATOMIC,
- .sk_flags = (1UL << SOCK_USE_WRITE_QUEUE),
- },
- .pmtudisc = IP_PMTUDISC_WANT,
- .uc_ttl = -1,
-};
-
-void ip_send_unicast_reply(struct net *net, struct sk_buff *skb,
+void ip_send_unicast_reply(struct sock *sk, struct sk_buff *skb,
const struct ip_options *sopt,
__be32 daddr, __be32 saddr,
const struct ip_reply_arg *arg,
struct ipcm_cookie ipc;
struct flowi4 fl4;
struct rtable *rt = skb_rtable(skb);
+ struct net *net = sock_net(sk);
struct sk_buff *nskb;
- struct sock *sk;
- struct inet_sock *inet;
int err;
if (__ip_options_echo(&replyopts.opt.opt, skb, sopt))
if (IS_ERR(rt))
return;
- inet = &get_cpu_var(unicast_sock);
+ inet_sk(sk)->tos = arg->tos;
- inet->tos = arg->tos;
- sk = &inet->sk;
sk->sk_priority = skb->priority;
sk->sk_protocol = ip_hdr(skb)->protocol;
sk->sk_bound_dev_if = arg->bound_dev_if;
- sock_net_set(sk, net);
- __skb_queue_head_init(&sk->sk_write_queue);
sk->sk_sndbuf = sysctl_wmem_default;
err = ip_append_data(sk, &fl4, ip_reply_glue_bits, arg->iov->iov_base,
len, 0, &ipc, &rt, MSG_DONTWAIT);
arg->csumoffset) = csum_fold(csum_add(nskb->csum,
arg->csum));
nskb->ip_summed = CHECKSUM_NONE;
- skb_orphan(nskb);
skb_set_queue_mapping(nskb, skb_get_queue_mapping(skb));
ip_push_pending_frames(sk, &fl4);
}
out:
- put_cpu_var(unicast_sock);
-
ip_rt_put(rt);
}
if (dst->dev->mtu < mtu)
return;
+ if (rt->rt_pmtu && rt->rt_pmtu < mtu)
+ return;
+
if (mtu < ip_rt_min_pmtu)
mtu = ip_rt_min_pmtu;
tcp_slow_start(tp, acked);
else {
bictcp_update(ca, tp->snd_cwnd);
- tcp_cong_avoid_ai(tp, ca->cnt);
+ tcp_cong_avoid_ai(tp, ca->cnt, 1);
}
}
* ABC caps N to 2. Slow start exits when cwnd grows over ssthresh and
* returns the leftover acks to adjust cwnd in congestion avoidance mode.
*/
-void tcp_slow_start(struct tcp_sock *tp, u32 acked)
+u32 tcp_slow_start(struct tcp_sock *tp, u32 acked)
{
u32 cwnd = tp->snd_cwnd + acked;
if (cwnd > tp->snd_ssthresh)
cwnd = tp->snd_ssthresh + 1;
+ acked -= cwnd - tp->snd_cwnd;
tp->snd_cwnd = min(cwnd, tp->snd_cwnd_clamp);
+
+ return acked;
}
EXPORT_SYMBOL_GPL(tcp_slow_start);
-/* In theory this is tp->snd_cwnd += 1 / tp->snd_cwnd (or alternative w) */
-void tcp_cong_avoid_ai(struct tcp_sock *tp, u32 w)
+/* In theory this is tp->snd_cwnd += 1 / tp->snd_cwnd (or alternative w),
+ * for every packet that was ACKed.
+ */
+void tcp_cong_avoid_ai(struct tcp_sock *tp, u32 w, u32 acked)
{
+ tp->snd_cwnd_cnt += acked;
if (tp->snd_cwnd_cnt >= w) {
- if (tp->snd_cwnd < tp->snd_cwnd_clamp)
- tp->snd_cwnd++;
- tp->snd_cwnd_cnt = 0;
- } else {
- tp->snd_cwnd_cnt++;
+ u32 delta = tp->snd_cwnd_cnt / w;
+
+ tp->snd_cwnd_cnt -= delta * w;
+ tp->snd_cwnd += delta;
}
+ tp->snd_cwnd = min(tp->snd_cwnd, tp->snd_cwnd_clamp);
}
EXPORT_SYMBOL_GPL(tcp_cong_avoid_ai);
return;
/* In "safe" area, increase. */
- if (tp->snd_cwnd <= tp->snd_ssthresh)
- tcp_slow_start(tp, acked);
+ if (tp->snd_cwnd <= tp->snd_ssthresh) {
+ acked = tcp_slow_start(tp, acked);
+ if (!acked)
+ return;
+ }
/* In dangerous area, increase slowly. */
- else
- tcp_cong_avoid_ai(tp, tp->snd_cwnd);
+ tcp_cong_avoid_ai(tp, tp->snd_cwnd, acked);
}
EXPORT_SYMBOL_GPL(tcp_reno_cong_avoid);
u32 epoch_start; /* beginning of an epoch */
u32 ack_cnt; /* number of acks */
u32 tcp_cwnd; /* estimated tcp cwnd */
-#define ACK_RATIO_SHIFT 4
-#define ACK_RATIO_LIMIT (32u << ACK_RATIO_SHIFT)
- u16 delayed_ack; /* estimate the ratio of Packets/ACKs << 4 */
+ u16 unused;
u8 sample_cnt; /* number of samples to decide curr_rtt */
u8 found; /* the exit point is found? */
u32 round_start; /* beginning of each round */
ca->bic_K = 0;
ca->delay_min = 0;
ca->epoch_start = 0;
- ca->delayed_ack = 2 << ACK_RATIO_SHIFT;
ca->ack_cnt = 0;
ca->tcp_cwnd = 0;
ca->found = 0;
/*
* Compute congestion window to use.
*/
-static inline void bictcp_update(struct bictcp *ca, u32 cwnd)
+static inline void bictcp_update(struct bictcp *ca, u32 cwnd, u32 acked)
{
u32 delta, bic_target, max_cnt;
u64 offs, t;
- ca->ack_cnt++; /* count the number of ACKs */
+ ca->ack_cnt += acked; /* count the number of ACKed packets */
if (ca->last_cwnd == cwnd &&
(s32)(tcp_time_stamp - ca->last_time) <= HZ / 32)
return;
+ /* The CUBIC function can update ca->cnt at most once per jiffy.
+ * On all cwnd reduction events, ca->epoch_start is set to 0,
+ * which will force a recalculation of ca->cnt.
+ */
+ if (ca->epoch_start && tcp_time_stamp == ca->last_time)
+ goto tcp_friendliness;
+
ca->last_cwnd = cwnd;
ca->last_time = tcp_time_stamp;
if (ca->epoch_start == 0) {
ca->epoch_start = tcp_time_stamp; /* record beginning */
- ca->ack_cnt = 1; /* start counting */
+ ca->ack_cnt = acked; /* start counting */
ca->tcp_cwnd = cwnd; /* syn with cubic */
if (ca->last_max_cwnd <= cwnd) {
if (ca->last_max_cwnd == 0 && ca->cnt > 20)
ca->cnt = 20; /* increase cwnd 5% per RTT */
+tcp_friendliness:
/* TCP Friendly */
if (tcp_friendliness) {
u32 scale = beta_scale;
}
}
- ca->cnt = (ca->cnt << ACK_RATIO_SHIFT) / ca->delayed_ack;
if (ca->cnt == 0) /* cannot be zero */
ca->cnt = 1;
}
if (tp->snd_cwnd <= tp->snd_ssthresh) {
if (hystart && after(ack, ca->end_seq))
bictcp_hystart_reset(sk);
- tcp_slow_start(tp, acked);
- } else {
- bictcp_update(ca, tp->snd_cwnd);
- tcp_cong_avoid_ai(tp, ca->cnt);
+ acked = tcp_slow_start(tp, acked);
+ if (!acked)
+ return;
}
+ bictcp_update(ca, tp->snd_cwnd, acked);
+ tcp_cong_avoid_ai(tp, ca->cnt, acked);
}
static u32 bictcp_recalc_ssthresh(struct sock *sk)
*/
static void bictcp_acked(struct sock *sk, u32 cnt, s32 rtt_us)
{
- const struct inet_connection_sock *icsk = inet_csk(sk);
const struct tcp_sock *tp = tcp_sk(sk);
struct bictcp *ca = inet_csk_ca(sk);
u32 delay;
- if (icsk->icsk_ca_state == TCP_CA_Open) {
- u32 ratio = ca->delayed_ack;
-
- ratio -= ca->delayed_ack >> ACK_RATIO_SHIFT;
- ratio += cnt;
-
- ca->delayed_ack = clamp(ratio, 1U, ACK_RATIO_LIMIT);
- }
-
/* Some calls are for duplicates without timetamps */
if (rtt_us < 0)
return;
arg.bound_dev_if = sk->sk_bound_dev_if;
arg.tos = ip_hdr(skb)->tos;
- ip_send_unicast_reply(net, skb, &TCP_SKB_CB(skb)->header.h4.opt,
+ ip_send_unicast_reply(*this_cpu_ptr(net->ipv4.tcp_sk),
+ skb, &TCP_SKB_CB(skb)->header.h4.opt,
ip_hdr(skb)->saddr, ip_hdr(skb)->daddr,
&arg, arg.iov[0].iov_len);
if (oif)
arg.bound_dev_if = oif;
arg.tos = tos;
- ip_send_unicast_reply(net, skb, &TCP_SKB_CB(skb)->header.h4.opt,
+ ip_send_unicast_reply(*this_cpu_ptr(net->ipv4.tcp_sk),
+ skb, &TCP_SKB_CB(skb)->header.h4.opt,
ip_hdr(skb)->saddr, ip_hdr(skb)->daddr,
&arg, arg.iov[0].iov_len);
};
EXPORT_SYMBOL(tcp_prot);
+static void __net_exit tcp_sk_exit(struct net *net)
+{
+ int cpu;
+
+ for_each_possible_cpu(cpu)
+ inet_ctl_sock_destroy(*per_cpu_ptr(net->ipv4.tcp_sk, cpu));
+ free_percpu(net->ipv4.tcp_sk);
+}
+
static int __net_init tcp_sk_init(struct net *net)
{
+ int res, cpu;
+
+ net->ipv4.tcp_sk = alloc_percpu(struct sock *);
+ if (!net->ipv4.tcp_sk)
+ return -ENOMEM;
+
+ for_each_possible_cpu(cpu) {
+ struct sock *sk;
+
+ res = inet_ctl_sock_create(&sk, PF_INET, SOCK_RAW,
+ IPPROTO_TCP, net);
+ if (res)
+ goto fail;
+ *per_cpu_ptr(net->ipv4.tcp_sk, cpu) = sk;
+ }
net->ipv4.sysctl_tcp_ecn = 2;
return 0;
-}
-static void __net_exit tcp_sk_exit(struct net *net)
-{
+fail:
+ tcp_sk_exit(net);
+
+ return res;
}
static void __net_exit tcp_sk_exit_batch(struct list_head *net_exit_list)
if (tp->snd_cwnd <= tp->snd_ssthresh)
tcp_slow_start(tp, acked);
else
- tcp_cong_avoid_ai(tp, min(tp->snd_cwnd, TCP_SCALABLE_AI_CNT));
+ tcp_cong_avoid_ai(tp, min(tp->snd_cwnd, TCP_SCALABLE_AI_CNT),
+ 1);
}
static u32 tcp_scalable_ssthresh(struct sock *sk)
/* In the "non-congestive state", increase cwnd
* every rtt.
*/
- tcp_cong_avoid_ai(tp, tp->snd_cwnd);
+ tcp_cong_avoid_ai(tp, tp->snd_cwnd, 1);
} else {
/* In the "congestive state", increase cwnd
* every other rtt.
} else {
/* Reno */
- tcp_cong_avoid_ai(tp, tp->snd_cwnd);
+ tcp_cong_avoid_ai(tp, tp->snd_cwnd, 1);
}
/* The key players are v_vegas.beg_snd_una and v_beg_snd_nxt.
if (code == ICMPV6_HDR_FIELD)
teli = ip6_tnl_parse_tlv_enc_lim(skb, skb->data);
- if (teli && teli == info - 2) {
+ if (teli && teli == be32_to_cpu(info) - 2) {
tel = (struct ipv6_tlv_tnl_enc_lim *) &skb->data[teli];
if (tel->encap_limit == 0) {
net_warn_ratelimited("%s: Too small encapsulation limit or routing loop in tunnel!\n",
}
break;
case ICMPV6_PKT_TOOBIG:
- mtu = info - offset;
+ mtu = be32_to_cpu(info) - offset;
if (mtu < IPV6_MIN_MTU)
mtu = IPV6_MIN_MTU;
t->dev->mtu = mtu;
skb_copy_secmark(to, from);
}
-static void ipv6_select_ident(struct frag_hdr *fhdr, struct rt6_info *rt)
-{
- static u32 ip6_idents_hashrnd __read_mostly;
- u32 hash, id;
-
- net_get_random_once(&ip6_idents_hashrnd, sizeof(ip6_idents_hashrnd));
-
- hash = __ipv6_addr_jhash(&rt->rt6i_dst.addr, ip6_idents_hashrnd);
- hash = __ipv6_addr_jhash(&rt->rt6i_src.addr, hash);
-
- id = ip_idents_reserve(hash, 1);
- fhdr->identification = htonl(id);
-}
-
int ip6_fragment(struct sk_buff *skb, int (*output)(struct sk_buff *))
{
struct sk_buff *frag;
#include <net/addrconf.h>
#include <net/secure_seq.h>
+u32 __ipv6_select_ident(u32 hashrnd, struct in6_addr *dst, struct in6_addr *src)
+{
+ u32 hash, id;
+
+ hash = __ipv6_addr_jhash(dst, hashrnd);
+ hash = __ipv6_addr_jhash(src, hash);
+
+ /* Treat id of 0 as unset and if we get 0 back from ip_idents_reserve,
+ * set the hight order instead thus minimizing possible future
+ * collisions.
+ */
+ id = ip_idents_reserve(hash, 1);
+ if (unlikely(!id))
+ id = 1 << 31;
+
+ return id;
+}
+
/* This function exists only for tap drivers that must support broken
* clients requesting UFO without specifying an IPv6 fragment ID.
*
static u32 ip6_proxy_idents_hashrnd __read_mostly;
struct in6_addr buf[2];
struct in6_addr *addrs;
- u32 hash, id;
+ u32 id;
addrs = skb_header_pointer(skb,
skb_network_offset(skb) +
net_get_random_once(&ip6_proxy_idents_hashrnd,
sizeof(ip6_proxy_idents_hashrnd));
- hash = __ipv6_addr_jhash(&addrs[1], ip6_proxy_idents_hashrnd);
- hash = __ipv6_addr_jhash(&addrs[0], hash);
-
- id = ip_idents_reserve(hash, 1);
- skb_shinfo(skb)->ip6_frag_id = htonl(id);
+ id = __ipv6_select_ident(ip6_proxy_idents_hashrnd,
+ &addrs[1], &addrs[0]);
+ skb_shinfo(skb)->ip6_frag_id = id;
}
EXPORT_SYMBOL_GPL(ipv6_proxy_select_ident);
+void ipv6_select_ident(struct frag_hdr *fhdr, struct rt6_info *rt)
+{
+ static u32 ip6_idents_hashrnd __read_mostly;
+ u32 id;
+
+ net_get_random_once(&ip6_idents_hashrnd, sizeof(ip6_idents_hashrnd));
+
+ id = __ipv6_select_ident(ip6_idents_hashrnd, &rt->rt6i_dst.addr,
+ &rt->rt6i_src.addr);
+ fhdr->identification = htonl(id);
+}
+EXPORT_SYMBOL(ipv6_select_ident);
+
int ip6_find_1stfragopt(struct sk_buff *skb, u8 **nexthdr)
{
u16 offset = sizeof(struct ipv6hdr);
if (data[IFLA_IPTUN_ENCAP_SPORT]) {
ret = true;
- ipencap->sport = nla_get_u16(data[IFLA_IPTUN_ENCAP_SPORT]);
+ ipencap->sport = nla_get_be16(data[IFLA_IPTUN_ENCAP_SPORT]);
}
if (data[IFLA_IPTUN_ENCAP_DPORT]) {
ret = true;
- ipencap->dport = nla_get_u16(data[IFLA_IPTUN_ENCAP_DPORT]);
+ ipencap->dport = nla_get_be16(data[IFLA_IPTUN_ENCAP_DPORT]);
}
return ret;
if (nla_put_u16(skb, IFLA_IPTUN_ENCAP_TYPE,
tunnel->encap.type) ||
- nla_put_u16(skb, IFLA_IPTUN_ENCAP_SPORT,
+ nla_put_be16(skb, IFLA_IPTUN_ENCAP_SPORT,
tunnel->encap.sport) ||
- nla_put_u16(skb, IFLA_IPTUN_ENCAP_DPORT,
+ nla_put_be16(skb, IFLA_IPTUN_ENCAP_DPORT,
tunnel->encap.dport) ||
nla_put_u16(skb, IFLA_IPTUN_ENCAP_FLAGS,
tunnel->encap.flags))
skb_shinfo(skb)->gso_segs = DIV_ROUND_UP(skb->len, mss);
+ /* Set the IPv6 fragment id if not set yet */
+ if (!skb_shinfo(skb)->ip6_frag_id)
+ ipv6_proxy_select_ident(skb);
+
segs = NULL;
goto out;
}
fptr = (struct frag_hdr *)(skb_network_header(skb) + unfrag_ip6hlen);
fptr->nexthdr = nexthdr;
fptr->reserved = 0;
- fptr->identification = skb_shinfo(skb)->ip6_frag_id;
+ if (skb_shinfo(skb)->ip6_frag_id)
+ fptr->identification = skb_shinfo(skb)->ip6_frag_id;
+ else
+ ipv6_select_ident(fptr,
+ (struct rt6_info *)skb_dst(skb));
/* Fragment the skb. ipv6 header and the remaining fields of the
* fragment header are updated in ipv6_gso_segment()
return err;
}
-static int ip_vs_route_me_harder(int af, struct sk_buff *skb)
+static int ip_vs_route_me_harder(int af, struct sk_buff *skb,
+ unsigned int hooknum)
{
+ if (!sysctl_snat_reroute(skb))
+ return 0;
+ /* Reroute replies only to remote clients (FORWARD and LOCAL_OUT) */
+ if (NF_INET_LOCAL_IN == hooknum)
+ return 0;
#ifdef CONFIG_IP_VS_IPV6
if (af == AF_INET6) {
- if (sysctl_snat_reroute(skb) && ip6_route_me_harder(skb) != 0)
+ struct dst_entry *dst = skb_dst(skb);
+
+ if (dst->dev && !(dst->dev->flags & IFF_LOOPBACK) &&
+ ip6_route_me_harder(skb) != 0)
return 1;
} else
#endif
- if ((sysctl_snat_reroute(skb) ||
- skb_rtable(skb)->rt_flags & RTCF_LOCAL) &&
+ if (!(skb_rtable(skb)->rt_flags & RTCF_LOCAL) &&
ip_route_me_harder(skb, RTN_LOCAL) != 0)
return 1;
union nf_inet_addr *snet,
__u8 protocol, struct ip_vs_conn *cp,
struct ip_vs_protocol *pp,
- unsigned int offset, unsigned int ihl)
+ unsigned int offset, unsigned int ihl,
+ unsigned int hooknum)
{
unsigned int verdict = NF_DROP;
#endif
ip_vs_nat_icmp(skb, pp, cp, 1);
- if (ip_vs_route_me_harder(af, skb))
+ if (ip_vs_route_me_harder(af, skb, hooknum))
goto out;
/* do the statistics and put it back */
snet.ip = iph->saddr;
return handle_response_icmp(AF_INET, skb, &snet, cih->protocol, cp,
- pp, ciph.len, ihl);
+ pp, ciph.len, ihl, hooknum);
}
#ifdef CONFIG_IP_VS_IPV6
snet.in6 = ciph.saddr.in6;
writable = ciph.len;
return handle_response_icmp(AF_INET6, skb, &snet, ciph.protocol, cp,
- pp, writable, sizeof(struct ipv6hdr));
+ pp, writable, sizeof(struct ipv6hdr),
+ hooknum);
}
#endif
*/
static unsigned int
handle_response(int af, struct sk_buff *skb, struct ip_vs_proto_data *pd,
- struct ip_vs_conn *cp, struct ip_vs_iphdr *iph)
+ struct ip_vs_conn *cp, struct ip_vs_iphdr *iph,
+ unsigned int hooknum)
{
struct ip_vs_protocol *pp = pd->pp;
* if it came from this machine itself. So re-compute
* the routing information.
*/
- if (ip_vs_route_me_harder(af, skb))
+ if (ip_vs_route_me_harder(af, skb, hooknum))
goto drop;
IP_VS_DBG_PKT(10, af, pp, skb, 0, "After SNAT");
cp = pp->conn_out_get(af, skb, &iph, 0);
if (likely(cp))
- return handle_response(af, skb, pd, cp, &iph);
+ return handle_response(af, skb, pd, cp, &iph, hooknum);
if (sysctl_nat_icmp_send(net) &&
(pp->protocol == IPPROTO_TCP ||
pp->protocol == IPPROTO_UDP ||
/* Restore old counters on this cpu, no problem. Per-cpu statistics
* are not exposed to userspace.
*/
+ preempt_disable();
stats = this_cpu_ptr(newstats);
stats->bytes = be64_to_cpu(nla_get_be64(tb[NFTA_COUNTER_BYTES]));
stats->pkts = be64_to_cpu(nla_get_be64(tb[NFTA_COUNTER_PACKETS]));
+ preempt_enable();
return newstats;
}
nft_ctx_init(&ctx, skb, nlh, afi, table, chain, nla);
trans = nft_trans_alloc(&ctx, NFT_MSG_NEWCHAIN,
sizeof(struct nft_trans_chain));
- if (trans == NULL)
+ if (trans == NULL) {
+ free_percpu(stats);
return -ENOMEM;
+ }
nft_trans_chain_stats(trans) = stats;
nft_trans_chain_update(trans) = true;
hookfn = type->hooks[hooknum];
basechain = kzalloc(sizeof(*basechain), GFP_KERNEL);
- if (basechain == NULL)
+ if (basechain == NULL) {
+ module_put(type->owner);
return -ENOMEM;
+ }
if (nla[NFTA_CHAIN_COUNTERS]) {
stats = nft_stats_alloc(nla[NFTA_CHAIN_COUNTERS]);
}
EXPORT_SYMBOL_GPL(nft_chain_validate_dependency);
+int nft_chain_validate_hooks(const struct nft_chain *chain,
+ unsigned int hook_flags)
+{
+ struct nft_base_chain *basechain;
+
+ if (chain->flags & NFT_BASE_CHAIN) {
+ basechain = nft_base_chain(chain);
+
+ if ((1 << basechain->ops[0].hooknum) & hook_flags)
+ return 0;
+
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(nft_chain_validate_hooks);
+
/*
* Loop detection - walk through the ruleset beginning at the destination chain
* of a new jump until either the source chain is reached (loop) or all
};
EXPORT_SYMBOL_GPL(nft_masq_policy);
+int nft_masq_validate(const struct nft_ctx *ctx,
+ const struct nft_expr *expr,
+ const struct nft_data **data)
+{
+ int err;
+
+ err = nft_chain_validate_dependency(ctx->chain, NFT_CHAIN_T_NAT);
+ if (err < 0)
+ return err;
+
+ return nft_chain_validate_hooks(ctx->chain,
+ (1 << NF_INET_POST_ROUTING));
+}
+EXPORT_SYMBOL_GPL(nft_masq_validate);
+
int nft_masq_init(const struct nft_ctx *ctx,
const struct nft_expr *expr,
const struct nlattr * const tb[])
struct nft_masq *priv = nft_expr_priv(expr);
int err;
- err = nft_chain_validate_dependency(ctx->chain, NFT_CHAIN_T_NAT);
- if (err < 0)
+ err = nft_masq_validate(ctx, expr, NULL);
+ if (err)
return err;
if (tb[NFTA_MASQ_FLAGS] == NULL)
}
EXPORT_SYMBOL_GPL(nft_masq_dump);
-int nft_masq_validate(const struct nft_ctx *ctx, const struct nft_expr *expr,
- const struct nft_data **data)
-{
- return nft_chain_validate_dependency(ctx->chain, NFT_CHAIN_T_NAT);
-}
-EXPORT_SYMBOL_GPL(nft_masq_validate);
-
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Arturo Borrero Gonzalez <arturo.borrero.glez@gmail.com>");
[NFTA_NAT_FLAGS] = { .type = NLA_U32 },
};
-static int nft_nat_init(const struct nft_ctx *ctx, const struct nft_expr *expr,
- const struct nlattr * const tb[])
+static int nft_nat_validate(const struct nft_ctx *ctx,
+ const struct nft_expr *expr,
+ const struct nft_data **data)
{
struct nft_nat *priv = nft_expr_priv(expr);
- u32 family;
int err;
err = nft_chain_validate_dependency(ctx->chain, NFT_CHAIN_T_NAT);
if (err < 0)
return err;
+ switch (priv->type) {
+ case NFT_NAT_SNAT:
+ err = nft_chain_validate_hooks(ctx->chain,
+ (1 << NF_INET_POST_ROUTING) |
+ (1 << NF_INET_LOCAL_IN));
+ break;
+ case NFT_NAT_DNAT:
+ err = nft_chain_validate_hooks(ctx->chain,
+ (1 << NF_INET_PRE_ROUTING) |
+ (1 << NF_INET_LOCAL_OUT));
+ break;
+ }
+
+ return err;
+}
+
+static int nft_nat_init(const struct nft_ctx *ctx, const struct nft_expr *expr,
+ const struct nlattr * const tb[])
+{
+ struct nft_nat *priv = nft_expr_priv(expr);
+ u32 family;
+ int err;
+
if (tb[NFTA_NAT_TYPE] == NULL ||
(tb[NFTA_NAT_REG_ADDR_MIN] == NULL &&
tb[NFTA_NAT_REG_PROTO_MIN] == NULL))
return -EINVAL;
}
+ err = nft_nat_validate(ctx, expr, NULL);
+ if (err < 0)
+ return err;
+
if (tb[NFTA_NAT_FAMILY] == NULL)
return -EINVAL;
return -1;
}
-static int nft_nat_validate(const struct nft_ctx *ctx,
- const struct nft_expr *expr,
- const struct nft_data **data)
-{
- return nft_chain_validate_dependency(ctx->chain, NFT_CHAIN_T_NAT);
-}
-
static struct nft_expr_type nft_nat_type;
static const struct nft_expr_ops nft_nat_ops = {
.type = &nft_nat_type,
};
EXPORT_SYMBOL_GPL(nft_redir_policy);
+int nft_redir_validate(const struct nft_ctx *ctx,
+ const struct nft_expr *expr,
+ const struct nft_data **data)
+{
+ int err;
+
+ err = nft_chain_validate_dependency(ctx->chain, NFT_CHAIN_T_NAT);
+ if (err < 0)
+ return err;
+
+ return nft_chain_validate_hooks(ctx->chain,
+ (1 << NF_INET_PRE_ROUTING) |
+ (1 << NF_INET_LOCAL_OUT));
+}
+EXPORT_SYMBOL_GPL(nft_redir_validate);
+
int nft_redir_init(const struct nft_ctx *ctx,
const struct nft_expr *expr,
const struct nlattr * const tb[])
struct nft_redir *priv = nft_expr_priv(expr);
int err;
- err = nft_chain_validate_dependency(ctx->chain, NFT_CHAIN_T_NAT);
+ err = nft_redir_validate(ctx, expr, NULL);
if (err < 0)
return err;
}
EXPORT_SYMBOL_GPL(nft_redir_dump);
-int nft_redir_validate(const struct nft_ctx *ctx, const struct nft_expr *expr,
- const struct nft_data **data)
-{
- return nft_chain_validate_dependency(ctx->chain, NFT_CHAIN_T_NAT);
-}
-EXPORT_SYMBOL_GPL(nft_redir_validate);
-
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Arturo Borrero Gonzalez <arturo.borrero.glez@gmail.com>");
for (undo = 0; undo < group; undo++)
if (test_bit(undo, &groups))
- nlk->netlink_unbind(sock_net(sk), undo);
+ nlk->netlink_unbind(sock_net(sk), undo + 1);
}
static int netlink_bind(struct socket *sock, struct sockaddr *addr,
for (group = 0; group < nlk->ngroups; group++) {
if (!test_bit(group, &groups))
continue;
- err = nlk->netlink_bind(net, group);
+ err = nlk->netlink_bind(net, group + 1);
if (!err)
continue;
netlink_undo_bind(group, groups, sk);
{
.procname = "max_unacked_packets",
.data = &rds_sysctl_max_unacked_packets,
- .maxlen = sizeof(unsigned long),
+ .maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec,
},
{
.procname = "max_unacked_bytes",
.data = &rds_sysctl_max_unacked_bytes,
- .maxlen = sizeof(unsigned long),
+ .maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec,
},
}
EXPORT_SYMBOL(tcf_exts_change);
-#define tcf_exts_first_act(ext) \
- list_first_entry(&(exts)->actions, struct tc_action, list)
+#define tcf_exts_first_act(ext) \
+ list_first_entry_or_null(&(exts)->actions, \
+ struct tc_action, list)
int tcf_exts_dump(struct sk_buff *skb, struct tcf_exts *exts)
{
{
#ifdef CONFIG_NET_CLS_ACT
struct tc_action *a = tcf_exts_first_act(exts);
- if (tcf_action_copy_stats(skb, a, 1) < 0)
+ if (a != NULL && tcf_action_copy_stats(skb, a, 1) < 0)
return -1;
#endif
return 0;
if (tb[TCA_FQ_FLOW_PLIMIT])
q->flow_plimit = nla_get_u32(tb[TCA_FQ_FLOW_PLIMIT]);
- if (tb[TCA_FQ_QUANTUM])
- q->quantum = nla_get_u32(tb[TCA_FQ_QUANTUM]);
+ if (tb[TCA_FQ_QUANTUM]) {
+ u32 quantum = nla_get_u32(tb[TCA_FQ_QUANTUM]);
+
+ if (quantum > 0)
+ q->quantum = quantum;
+ else
+ err = -EINVAL;
+ }
if (tb[TCA_FQ_INITIAL_QUANTUM])
q->initial_quantum = nla_get_u32(tb[TCA_FQ_INITIAL_QUANTUM]);
addr_param = param.v + sizeof(sctp_addip_param_t);
- af = sctp_get_af_specific(param_type2af(param.p->type));
+ af = sctp_get_af_specific(param_type2af(addr_param->p.type));
if (af == NULL)
break;
select SECURITYFS
select SECURITY_PATH
select SECURITY_NETWORK
+ select SRCU
default n
help
This selects TOMOYO Linux, pathname-based access control.
static int my_client = -1;
-/*
- * unuse callback - send ALL_SOUNDS_OFF and RESET_CONTROLLERS events
- * to subscribers.
- * Note: this callback is called only after all subscribers are removed.
- */
-static int
-dummy_unuse(void *private_data, struct snd_seq_port_subscribe *info)
-{
- struct snd_seq_dummy_port *p;
- int i;
- struct snd_seq_event ev;
-
- p = private_data;
- memset(&ev, 0, sizeof(ev));
- if (p->duplex)
- ev.source.port = p->connect;
- else
- ev.source.port = p->port;
- ev.dest.client = SNDRV_SEQ_ADDRESS_SUBSCRIBERS;
- ev.type = SNDRV_SEQ_EVENT_CONTROLLER;
- for (i = 0; i < 16; i++) {
- ev.data.control.channel = i;
- ev.data.control.param = MIDI_CTL_ALL_SOUNDS_OFF;
- snd_seq_kernel_client_dispatch(p->client, &ev, 0, 0);
- ev.data.control.param = MIDI_CTL_RESET_CONTROLLERS;
- snd_seq_kernel_client_dispatch(p->client, &ev, 0, 0);
- }
- return 0;
-}
-
/*
* event input callback - just redirect events to subscribers
*/
| SNDRV_SEQ_PORT_TYPE_PORT;
memset(&pcb, 0, sizeof(pcb));
pcb.owner = THIS_MODULE;
- pcb.unuse = dummy_unuse;
pcb.event_input = dummy_input;
pcb.private_free = dummy_free;
pcb.private_data = rec;
static void snd_ak4113_free(struct ak4113 *chip)
{
- chip->init = 1; /* don't schedule new work */
- mb();
+ atomic_inc(&chip->wq_processing); /* don't schedule new work */
cancel_delayed_work_sync(&chip->work);
kfree(chip);
}
chip->write = write;
chip->private_data = private_data;
INIT_DELAYED_WORK(&chip->work, ak4113_stats);
+ atomic_set(&chip->wq_processing, 0);
for (reg = 0; reg < AK4113_WRITABLE_REGS ; reg++)
chip->regmap[reg] = pgm[reg];
void snd_ak4113_reinit(struct ak4113 *chip)
{
- chip->init = 1;
- mb();
- flush_delayed_work(&chip->work);
+ if (atomic_inc_return(&chip->wq_processing) == 1)
+ cancel_delayed_work_sync(&chip->work);
ak4113_init_regs(chip);
/* bring up statistics / event queing */
- chip->init = 0;
- if (chip->kctls[0])
+ if (atomic_dec_and_test(&chip->wq_processing))
schedule_delayed_work(&chip->work, HZ / 10);
}
EXPORT_SYMBOL_GPL(snd_ak4113_reinit);
{
struct ak4113 *chip = container_of(work, struct ak4113, work.work);
- if (!chip->init)
+ if (atomic_inc_return(&chip->wq_processing) == 1)
snd_ak4113_check_rate_and_errors(chip, chip->check_flags);
- schedule_delayed_work(&chip->work, HZ / 10);
+ if (atomic_dec_and_test(&chip->wq_processing))
+ schedule_delayed_work(&chip->work, HZ / 10);
}
static void snd_ak4114_free(struct ak4114 *chip)
{
- chip->init = 1; /* don't schedule new work */
- mb();
+ atomic_inc(&chip->wq_processing); /* don't schedule new work */
cancel_delayed_work_sync(&chip->work);
kfree(chip);
}
chip->write = write;
chip->private_data = private_data;
INIT_DELAYED_WORK(&chip->work, ak4114_stats);
+ atomic_set(&chip->wq_processing, 0);
for (reg = 0; reg < 6; reg++)
chip->regmap[reg] = pgm[reg];
void snd_ak4114_reinit(struct ak4114 *chip)
{
- chip->init = 1;
- mb();
- flush_delayed_work(&chip->work);
+ if (atomic_inc_return(&chip->wq_processing) == 1)
+ cancel_delayed_work_sync(&chip->work);
ak4114_init_regs(chip);
/* bring up statistics / event queing */
- chip->init = 0;
- if (chip->kctls[0])
+ if (atomic_dec_and_test(&chip->wq_processing))
schedule_delayed_work(&chip->work, HZ / 10);
}
{
struct ak4114 *chip = container_of(work, struct ak4114, work.work);
- if (!chip->init)
+ if (atomic_inc_return(&chip->wq_processing) == 1)
snd_ak4114_check_rate_and_errors(chip, chip->check_flags);
-
- schedule_delayed_work(&chip->work, HZ / 10);
+ if (atomic_dec_and_test(&chip->wq_processing))
+ schedule_delayed_work(&chip->work, HZ / 10);
}
EXPORT_SYMBOL(snd_ak4114_create);
if (ret)
goto err_clk_disable;
+ return 0;
+
err_clk_disable:
clk_disable_unprepare(i2s->clk);
return ret;
struct atmel_pcm_dma_params *dma_params;
int dir, channels, bits;
u32 tfmr, rfmr, tcmr, rcmr;
- int start_event;
int ret;
int fslen, fslen_ext;
* The SSC transmit clock is obtained from the BCLK signal on
* on the TK line, and the SSC receive clock is
* generated from the transmit clock.
- *
- * For single channel data, one sample is transferred
- * on the falling edge of the LRC clock.
- * For two channel data, one sample is
- * transferred on both edges of the LRC clock.
*/
- start_event = ((channels == 1)
- ? SSC_START_FALLING_RF
- : SSC_START_EDGE_RF);
-
rcmr = SSC_BF(RCMR_PERIOD, 0)
| SSC_BF(RCMR_STTDLY, START_DELAY)
- | SSC_BF(RCMR_START, start_event)
+ | SSC_BF(RCMR_START, SSC_START_FALLING_RF)
| SSC_BF(RCMR_CKI, SSC_CKI_RISING)
| SSC_BF(RCMR_CKO, SSC_CKO_NONE)
| SSC_BF(RCMR_CKS, ssc->clk_from_rk_pin ?
rfmr = SSC_BF(RFMR_FSEDGE, SSC_FSEDGE_POSITIVE)
| SSC_BF(RFMR_FSOS, SSC_FSOS_NONE)
| SSC_BF(RFMR_FSLEN, 0)
- | SSC_BF(RFMR_DATNB, 0)
+ | SSC_BF(RFMR_DATNB, (channels - 1))
| SSC_BIT(RFMR_MSBF)
| SSC_BF(RFMR_LOOP, 0)
| SSC_BF(RFMR_DATLEN, (bits - 1));
tcmr = SSC_BF(TCMR_PERIOD, 0)
| SSC_BF(TCMR_STTDLY, START_DELAY)
- | SSC_BF(TCMR_START, start_event)
+ | SSC_BF(TCMR_START, SSC_START_FALLING_RF)
| SSC_BF(TCMR_CKI, SSC_CKI_FALLING)
| SSC_BF(TCMR_CKO, SSC_CKO_NONE)
| SSC_BF(TCMR_CKS, ssc->clk_from_rk_pin ?
| SSC_BF(TFMR_FSDEN, 0)
| SSC_BF(TFMR_FSOS, SSC_FSOS_NONE)
| SSC_BF(TFMR_FSLEN, 0)
- | SSC_BF(TFMR_DATNB, 0)
+ | SSC_BF(TFMR_DATNB, (channels - 1))
| SSC_BIT(TFMR_MSBF)
| SSC_BF(TFMR_DATDEF, 0)
| SSC_BF(TFMR_DATLEN, (bits - 1));
rcmr = SSC_BF(RCMR_PERIOD, ssc_p->rcmr_period)
| SSC_BF(RCMR_STTDLY, 1)
| SSC_BF(RCMR_START, SSC_START_RISING_RF)
- | SSC_BF(RCMR_CKI, SSC_CKI_RISING)
+ | SSC_BF(RCMR_CKI, SSC_CKI_FALLING)
| SSC_BF(RCMR_CKO, SSC_CKO_NONE)
| SSC_BF(RCMR_CKS, SSC_CKS_DIV);
tcmr = SSC_BF(TCMR_PERIOD, ssc_p->tcmr_period)
| SSC_BF(TCMR_STTDLY, 1)
| SSC_BF(TCMR_START, SSC_START_RISING_RF)
- | SSC_BF(TCMR_CKI, SSC_CKI_RISING)
+ | SSC_BF(TCMR_CKI, SSC_CKI_FALLING)
| SSC_BF(TCMR_CKO, SSC_CKO_CONTINUOUS)
| SSC_BF(TCMR_CKS, SSC_CKS_DIV);
rcmr = SSC_BF(RCMR_PERIOD, 0)
| SSC_BF(RCMR_STTDLY, START_DELAY)
| SSC_BF(RCMR_START, SSC_START_RISING_RF)
- | SSC_BF(RCMR_CKI, SSC_CKI_RISING)
+ | SSC_BF(RCMR_CKI, SSC_CKI_FALLING)
| SSC_BF(RCMR_CKO, SSC_CKO_NONE)
| SSC_BF(RCMR_CKS, ssc->clk_from_rk_pin ?
SSC_CKS_PIN : SSC_CKS_CLOCK);
static const char * const pcm512x_dsp_program_texts[] = {
"FIR interpolation with de-emphasis",
"Low latency IIR with de-emphasis",
- "Fixed process flow",
"High attenuation with de-emphasis",
+ "Fixed process flow",
"Ringing-less low latency FIR",
};
RT286_I2S_CTRL1, 0x0018, d_len_code << 3);
dev_dbg(codec->dev, "format val = 0x%x\n", val);
- if (substream->stream == SNDRV_PCM_STREAM_PLAYBACK)
- snd_soc_update_bits(codec, RT286_DAC_FORMAT, 0x407f, val);
- else
- snd_soc_update_bits(codec, RT286_ADC_FORMAT, 0x407f, val);
+ snd_soc_update_bits(codec, RT286_DAC_FORMAT, 0x407f, val);
+ snd_soc_update_bits(codec, RT286_ADC_FORMAT, 0x407f, val);
return 0;
}
static struct acpi_device_id rt5640_acpi_match[] = {
{ "INT33CA", 0 },
{ "10EC5640", 0 },
+ { "10EC5642", 0 },
{ },
};
MODULE_DEVICE_TABLE(acpi, rt5640_acpi_match);
struct rt5677_priv *rt5677 = snd_soc_codec_get_drvdata(codec);
switch (event) {
- case SND_SOC_DAPM_POST_PMU:
+ case SND_SOC_DAPM_PRE_PMU:
regmap_update_bits(rt5677->regmap, RT5677_PLL1_CTRL2, 0x2, 0x2);
+ break;
+
+ case SND_SOC_DAPM_POST_PMU:
regmap_update_bits(rt5677->regmap, RT5677_PLL1_CTRL2, 0x2, 0x0);
break;
+
default:
return 0;
}
struct rt5677_priv *rt5677 = snd_soc_codec_get_drvdata(codec);
switch (event) {
- case SND_SOC_DAPM_POST_PMU:
+ case SND_SOC_DAPM_PRE_PMU:
regmap_update_bits(rt5677->regmap, RT5677_PLL2_CTRL2, 0x2, 0x2);
+ break;
+
+ case SND_SOC_DAPM_POST_PMU:
regmap_update_bits(rt5677->regmap, RT5677_PLL2_CTRL2, 0x2, 0x0);
break;
+
default:
return 0;
}
static const struct snd_soc_dapm_widget rt5677_dapm_widgets[] = {
SND_SOC_DAPM_SUPPLY("PLL1", RT5677_PWR_ANLG2, RT5677_PWR_PLL1_BIT,
- 0, rt5677_set_pll1_event, SND_SOC_DAPM_POST_PMU),
+ 0, rt5677_set_pll1_event, SND_SOC_DAPM_PRE_PMU |
+ SND_SOC_DAPM_POST_PMU),
SND_SOC_DAPM_SUPPLY("PLL2", RT5677_PWR_ANLG2, RT5677_PWR_PLL2_BIT,
- 0, rt5677_set_pll2_event, SND_SOC_DAPM_POST_PMU),
+ 0, rt5677_set_pll2_event, SND_SOC_DAPM_PRE_PMU |
+ SND_SOC_DAPM_POST_PMU),
/* Input Side */
/* micbias */
/* setting i2s data format */
switch (fmt & SND_SOC_DAIFMT_FORMAT_MASK) {
case SND_SOC_DAIFMT_DSP_A:
- i2sctl |= SGTL5000_I2S_MODE_PCM;
+ i2sctl |= SGTL5000_I2S_MODE_PCM << SGTL5000_I2S_MODE_SHIFT;
break;
case SND_SOC_DAIFMT_DSP_B:
- i2sctl |= SGTL5000_I2S_MODE_PCM;
+ i2sctl |= SGTL5000_I2S_MODE_PCM << SGTL5000_I2S_MODE_SHIFT;
i2sctl |= SGTL5000_I2S_LRALIGN;
break;
case SND_SOC_DAIFMT_I2S:
- i2sctl |= SGTL5000_I2S_MODE_I2S_LJ;
+ i2sctl |= SGTL5000_I2S_MODE_I2S_LJ << SGTL5000_I2S_MODE_SHIFT;
break;
case SND_SOC_DAIFMT_RIGHT_J:
- i2sctl |= SGTL5000_I2S_MODE_RJ;
+ i2sctl |= SGTL5000_I2S_MODE_RJ << SGTL5000_I2S_MODE_SHIFT;
i2sctl |= SGTL5000_I2S_LRPOL;
break;
case SND_SOC_DAIFMT_LEFT_J:
- i2sctl |= SGTL5000_I2S_MODE_I2S_LJ;
+ i2sctl |= SGTL5000_I2S_MODE_I2S_LJ << SGTL5000_I2S_MODE_SHIFT;
i2sctl |= SGTL5000_I2S_LRALIGN;
break;
default:
if (ret)
return ret;
+ /* Need 8 clocks before I2C accesses */
+ udelay(1);
+
/* read chip information */
ret = regmap_read(sgtl5000->regmap, SGTL5000_CHIP_ID, ®);
if (ret)
delay += aic3x->tdm_delay;
/* Configure data delay */
- snd_soc_write(codec, AIC3X_ASD_INTF_CTRLC, aic3x->tdm_delay);
+ snd_soc_write(codec, AIC3X_ASD_INTF_CTRLC, delay);
return 0;
}
struct ts3a227e *ts3a227e;
struct device *dev = &i2c->dev;
int ret;
+ unsigned int acc_reg;
ts3a227e = devm_kzalloc(&i2c->dev, sizeof(*ts3a227e), GFP_KERNEL);
if (ts3a227e == NULL)
INTB_DISABLE | ADC_COMPLETE_INT_DISABLE,
ADC_COMPLETE_INT_DISABLE);
+ /* Read jack status because chip might not trigger interrupt at boot. */
+ regmap_read(ts3a227e->regmap, TS3A227E_REG_ACCESSORY_STATUS, &acc_reg);
+ ts3a227e_new_jack_state(ts3a227e, acc_reg);
+ ts3a227e_jack_report(ts3a227e);
+
return 0;
}
if (wm8731 == NULL)
return -ENOMEM;
+ mutex_init(&wm8731->lock);
+
wm8731->regmap = devm_regmap_init_i2c(i2c, &wm8731_regmap);
if (IS_ERR(wm8731->regmap)) {
ret = PTR_ERR(wm8731->regmap);
{ "Right Capture PGA", NULL, "Right Capture Mux" },
{ "Right Capture PGA", NULL, "Right Capture Inverting Mux" },
- { "AIFOUTL", "Left", "ADCL" },
- { "AIFOUTL", "Right", "ADCR" },
- { "AIFOUTR", "Left", "ADCL" },
- { "AIFOUTR", "Right", "ADCR" },
+ { "AIFOUTL Mux", "Left", "ADCL" },
+ { "AIFOUTL Mux", "Right", "ADCR" },
+ { "AIFOUTR Mux", "Left", "ADCL" },
+ { "AIFOUTR Mux", "Right", "ADCR" },
+
+ { "AIFOUTL", NULL, "AIFOUTL Mux" },
+ { "AIFOUTR", NULL, "AIFOUTR Mux" },
{ "ADCL", NULL, "CLK_DSP" },
{ "ADCL", NULL, "Left Capture PGA" },
};
static const struct snd_soc_dapm_route dac_intercon[] = {
- { "DACL", "Right", "AIFINR" },
- { "DACL", "Left", "AIFINL" },
+ { "DACL Mux", "Left", "AIFINL" },
+ { "DACL Mux", "Right", "AIFINR" },
+
+ { "DACR Mux", "Left", "AIFINL" },
+ { "DACR Mux", "Right", "AIFINR" },
+
+ { "DACL", NULL, "DACL Mux" },
{ "DACL", NULL, "CLK_DSP" },
- { "DACR", "Right", "AIFINR" },
- { "DACR", "Left", "AIFINL" },
+ { "DACR", NULL, "DACR Mux" },
{ "DACR", NULL, "CLK_DSP" },
{ "Charge pump", NULL, "SYSCLK" },
{ 22050, 2 },
{ 24000, 2 },
{ 16000, 3 },
- { 11250, 4 },
+ { 11025, 4 },
{ 12000, 4 },
{ 8000, 5 },
};
struct snd_ac97 *ac97;
int ret = 0;
- ac97 = snd_soc_new_ac97_codec(codec);
+ ac97 = snd_soc_alloc_ac97_codec(codec);
if (IS_ERR(ac97)) {
ret = PTR_ERR(ac97);
dev_err(codec->dev, "Failed to register AC97 codec\n");
return ret;
}
- snd_soc_codec_set_drvdata(codec, ac97);
-
ret = wm9705_reset(codec);
if (ret)
- goto reset_err;
+ goto err_put_device;
+
+ ret = device_add(&ac97->dev);
+ if (ret)
+ goto err_put_device;
+
+ snd_soc_codec_set_drvdata(codec, ac97);
return 0;
-reset_err:
- snd_soc_free_ac97_codec(ac97);
+err_put_device:
+ put_device(&ac97->dev);
return ret;
}
struct wm9712_priv *wm9712 = snd_soc_codec_get_drvdata(codec);
int ret = 0;
- wm9712->ac97 = snd_soc_new_ac97_codec(codec);
+ wm9712->ac97 = snd_soc_alloc_ac97_codec(codec);
if (IS_ERR(wm9712->ac97)) {
ret = PTR_ERR(wm9712->ac97);
dev_err(codec->dev, "Failed to register AC97 codec: %d\n", ret);
ret = wm9712_reset(codec, 0);
if (ret < 0)
- goto reset_err;
+ goto err_put_device;
+
+ ret = device_add(&wm9712->ac97->dev);
+ if (ret)
+ goto err_put_device;
/* set alc mux to none */
ac97_write(codec, AC97_VIDEO, ac97_read(codec, AC97_VIDEO) | 0x3000);
return 0;
-reset_err:
- snd_soc_free_ac97_codec(wm9712->ac97);
+err_put_device:
+ put_device(&wm9712->ac97->dev);
return ret;
}
struct wm9713_priv *wm9713 = snd_soc_codec_get_drvdata(codec);
int ret = 0, reg;
- wm9713->ac97 = snd_soc_new_ac97_codec(codec);
+ wm9713->ac97 = snd_soc_alloc_ac97_codec(codec);
if (IS_ERR(wm9713->ac97))
return PTR_ERR(wm9713->ac97);
wm9713_reset(codec, 0);
ret = wm9713_reset(codec, 1);
if (ret < 0)
- goto reset_err;
+ goto err_put_device;
+
+ ret = device_add(&wm9713->ac97->dev);
+ if (ret)
+ goto err_put_device;
/* unmute the adc - move to kcontrol */
reg = ac97_read(codec, AC97_CD) & 0x7fff;
return 0;
-reset_err:
- snd_soc_free_ac97_codec(wm9713->ac97);
+err_put_device:
+ put_device(&wm9713->ac97->dev);
return ret;
}
#define ESAI_xCCR_xFP_MASK (((1 << ESAI_xCCR_xFP_WIDTH) - 1) << ESAI_xCCR_xFP_SHIFT)
#define ESAI_xCCR_xFP(v) ((((v) - 1) << ESAI_xCCR_xFP_SHIFT) & ESAI_xCCR_xFP_MASK)
#define ESAI_xCCR_xDC_SHIFT 9
-#define ESAI_xCCR_xDC_WIDTH 4
+#define ESAI_xCCR_xDC_WIDTH 5
#define ESAI_xCCR_xDC_MASK (((1 << ESAI_xCCR_xDC_WIDTH) - 1) << ESAI_xCCR_xDC_SHIFT)
#define ESAI_xCCR_xDC(v) ((((v) - 1) << ESAI_xCCR_xDC_SHIFT) & ESAI_xCCR_xDC_MASK)
#define ESAI_xCCR_xPSR_SHIFT 8
}
ssi_private->irq = platform_get_irq(pdev, 0);
- if (!ssi_private->irq) {
+ if (ssi_private->irq < 0) {
dev_err(&pdev->dev, "no irq for node %s\n", np->full_name);
- return -ENXIO;
+ return ssi_private->irq;
}
/* Are the RX and the TX clocks locked? */
if (ret)
goto clk_fail;
data->card.num_links = 1;
+ data->card.owner = THIS_MODULE;
data->card.dai_link = &data->dai;
data->card.dapm_widgets = imx_wm8962_dapm_widgets;
data->card.num_dapm_widgets = ARRAY_SIZE(imx_wm8962_dapm_widgets);
}
/* Decrease the reference count of the device nodes */
-static int asoc_simple_card_unref(struct platform_device *pdev)
+static int asoc_simple_card_unref(struct snd_soc_card *card)
{
- struct snd_soc_card *card = platform_get_drvdata(pdev);
struct snd_soc_dai_link *dai_link;
int num_links;
return ret;
err:
- asoc_simple_card_unref(pdev);
+ asoc_simple_card_unref(&priv->snd_card);
return ret;
}
snd_soc_jack_free_gpios(&simple_card_mic_jack, 1,
&simple_card_mic_jack_gpio);
- return asoc_simple_card_unref(pdev);
+ return asoc_simple_card_unref(card);
}
static const struct of_device_id asoc_simple_of_match[] = {
struct list_head *block_list)
{
struct sst_mem_block *block, *tmp;
+ struct sst_block_allocator ba_tmp = *ba;
u32 end = ba->offset + ba->size, block_end;
int err;
if (ba->offset >= block->offset && ba->offset < block_end) {
/* align ba to block boundary */
- ba->size -= block_end - ba->offset;
- ba->offset = block_end;
- err = block_alloc_contiguous(dsp, ba, block_list);
+ ba_tmp.size -= block_end - ba->offset;
+ ba_tmp.offset = block_end;
+ err = block_alloc_contiguous(dsp, &ba_tmp, block_list);
if (err < 0)
return -ENOMEM;
list_move(&block->list, &dsp->used_block_list);
list_add(&block->module_list, block_list);
/* align ba to block boundary */
- ba->size -= block_end - ba->offset;
- ba->offset = block_end;
+ ba_tmp.size -= block_end - ba->offset;
+ ba_tmp.offset = block_end;
- err = block_alloc_contiguous(dsp, ba, block_list);
+ err = block_alloc_contiguous(dsp, &ba_tmp, block_list);
if (err < 0)
return -ENOMEM;
}
/* tell DSP that notification has been handled */
- sst_dsp_shim_update_bits_unlocked(hsw->dsp, SST_IPCD,
+ sst_dsp_shim_update_bits(hsw->dsp, SST_IPCD,
SST_IPCD_BUSY | SST_IPCD_DONE, SST_IPCD_DONE);
/* unmask busy interrupt */
- sst_dsp_shim_update_bits_unlocked(hsw->dsp, SST_IMRX, SST_IMRX_BUSY, 0);
+ sst_dsp_shim_update_bits(hsw->dsp, SST_IMRX, SST_IMRX_BUSY, 0);
}
static struct ipc_message *reply_find_msg(struct sst_hsw *hsw, u32 header)
struct sst_dsp *sst = hsw->dsp;
unsigned long flags;
+ if (!stream) {
+ dev_warn(hsw->dev, "warning: stream is NULL, no stream to free, ignore it.\n");
+ return 0;
+ }
+
/* dont free DSP streams that are not commited */
if (!stream->commited)
goto out;
u32 header;
int ret;
+ if (!stream) {
+ dev_warn(hsw->dev, "warning: stream is NULL, no stream to commit, ignore it.\n");
+ return 0;
+ }
+
+ if (stream->commited) {
+ dev_warn(hsw->dev, "warning: stream is already committed, ignore it.\n");
+ return 0;
+ }
+
trace_ipc_request("stream alloc", stream->host_id);
header = IPC_GLB_TYPE(IPC_GLB_ALLOCATE_STREAM);
{
int ret;
+ if (!stream) {
+ dev_warn(hsw->dev, "warning: stream is NULL, no stream to pause, ignore it.\n");
+ return 0;
+ }
+
trace_ipc_request("stream pause", stream->reply.stream_hw_id);
ret = sst_hsw_stream_operations(hsw, IPC_STR_PAUSE,
{
int ret;
+ if (!stream) {
+ dev_warn(hsw->dev, "warning: stream is NULL, no stream to resume, ignore it.\n");
+ return 0;
+ }
+
trace_ipc_request("stream resume", stream->reply.stream_hw_id);
ret = sst_hsw_stream_operations(hsw, IPC_STR_RESUME,
{
int ret, tries = 10;
+ if (!stream) {
+ dev_warn(hsw->dev, "warning: stream is NULL, no stream to reset, ignore it.\n");
+ return 0;
+ }
+
/* dont reset streams that are not commited */
if (!stream->commited)
return 0;
/* Cherryview-based platforms: CherryTrail and Braswell */
static struct sst_machines sst_acpi_chv[] = {
- {"10EC5670", "cht-bsw", "cht-bsw-rt5672", NULL, "fw_sst_22a8.bin",
+ {"10EC5670", "cht-bsw", "cht-bsw-rt5672", NULL, "intel/fw_sst_22a8.bin",
&chv_platform_data },
{},
};
case SND_SOC_DAIFMT_CBM_CFS:
/* McBSP slave. FS clock as output */
regs->srgr2 |= FSGM;
- regs->pcr0 |= FSXM;
+ regs->pcr0 |= FSXM | FSRM;
break;
case SND_SOC_DAIFMT_CBM_CFM:
/* McBSP slave */
SNDRV_PCM_FMTBIT_S24_LE),
},
.ops = &rockchip_i2s_dai_ops,
+ .symmetric_rates = 1,
};
static const struct snd_soc_component_driver rockchip_i2s_component = {
}
/**
- * snd_soc_new_ac97_codec - initailise AC97 device
- * @codec: audio codec
+ * snd_soc_alloc_ac97_codec() - Allocate new a AC'97 device
+ * @codec: The CODEC for which to create the AC'97 device
*
- * Initialises AC97 codec resources for use by ad-hoc devices only.
+ * Allocated a new snd_ac97 device and intializes it, but does not yet register
+ * it. The caller is responsible to either call device_add(&ac97->dev) to
+ * register the device, or to call put_device(&ac97->dev) to free the device.
+ *
+ * Returns: A snd_ac97 device or a PTR_ERR in case of an error.
*/
-struct snd_ac97 *snd_soc_new_ac97_codec(struct snd_soc_codec *codec)
+struct snd_ac97 *snd_soc_alloc_ac97_codec(struct snd_soc_codec *codec)
{
struct snd_ac97 *ac97;
- int ret;
ac97 = kzalloc(sizeof(struct snd_ac97), GFP_KERNEL);
if (ac97 == NULL)
codec->component.card->snd_card->number, 0,
codec->component.name);
- ret = device_register(&ac97->dev);
+ device_initialize(&ac97->dev);
+
+ return ac97;
+}
+EXPORT_SYMBOL(snd_soc_alloc_ac97_codec);
+
+/**
+ * snd_soc_new_ac97_codec - initailise AC97 device
+ * @codec: audio codec
+ *
+ * Initialises AC97 codec resources for use by ad-hoc devices only.
+ */
+struct snd_ac97 *snd_soc_new_ac97_codec(struct snd_soc_codec *codec)
+{
+ struct snd_ac97 *ac97;
+ int ret;
+
+ ac97 = snd_soc_alloc_ac97_codec(codec);
+ if (IS_ERR(ac97))
+ return ac97;
+
+ ret = device_add(&ac97->dev);
if (ret) {
put_device(&ac97->dev);
return ERR_PTR(ret);
rtd->dai_link->stream_name);
ret = snd_pcm_new_internal(rtd->card->snd_card, new_name, num,
- 1, 0, &be_pcm);
+ rtd->dai_link->dpcm_playback,
+ rtd->dai_link->dpcm_capture, &be_pcm);
if (ret < 0) {
dev_err(rtd->card->dev, "ASoC: can't create compressed for %s\n",
rtd->dai_link->name);
rtd->pcm = be_pcm;
rtd->fe_compr = 1;
- be_pcm->streams[SNDRV_PCM_STREAM_PLAYBACK].substream->private_data = rtd;
- be_pcm->streams[SNDRV_PCM_STREAM_CAPTURE].substream->private_data = rtd;
+ if (rtd->dai_link->dpcm_playback)
+ be_pcm->streams[SNDRV_PCM_STREAM_PLAYBACK].substream->private_data = rtd;
+ else if (rtd->dai_link->dpcm_capture)
+ be_pcm->streams[SNDRV_PCM_STREAM_CAPTURE].substream->private_data = rtd;
memcpy(compr->ops, &soc_compr_dyn_ops, sizeof(soc_compr_dyn_ops));
} else
memcpy(compr->ops, &soc_compr_ops, sizeof(soc_compr_ops));
+#define _GNU_SOURCE
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
out:
return debugfs_mountpoint;
}
+
+int debugfs__strerror_open(int err, char *buf, size_t size, const char *filename)
+{
+ char sbuf[128];
+
+ switch (err) {
+ case ENOENT:
+ if (debugfs_found) {
+ snprintf(buf, size,
+ "Error:\tFile %s/%s not found.\n"
+ "Hint:\tPerhaps this kernel misses some CONFIG_ setting to enable this feature?.\n",
+ debugfs_mountpoint, filename);
+ break;
+ }
+ snprintf(buf, size, "%s",
+ "Error:\tUnable to find debugfs\n"
+ "Hint:\tWas your kernel compiled with debugfs support?\n"
+ "Hint:\tIs the debugfs filesystem mounted?\n"
+ "Hint:\tTry 'sudo mount -t debugfs nodev /sys/kernel/debug'");
+ break;
+ case EACCES:
+ snprintf(buf, size,
+ "Error:\tNo permissions to read %s/%s\n"
+ "Hint:\tTry 'sudo mount -o remount,mode=755 %s'\n",
+ debugfs_mountpoint, filename, debugfs_mountpoint);
+ break;
+ default:
+ snprintf(buf, size, "%s", strerror_r(err, sbuf, sizeof(sbuf)));
+ break;
+ }
+
+ return 0;
+}
+
+int debugfs__strerror_open_tp(int err, char *buf, size_t size, const char *sys, const char *name)
+{
+ char path[PATH_MAX];
+
+ snprintf(path, PATH_MAX, "tracing/events/%s/%s", sys, name ?: "*");
+
+ return debugfs__strerror_open(err, buf, size, path);
+}
extern char debugfs_mountpoint[];
+int debugfs__strerror_open(int err, char *buf, size_t size, const char *filename);
+int debugfs__strerror_open_tp(int err, char *buf, size_t size, const char *sys, const char *name);
+
#endif /* __API_DEBUGFS_H__ */
--- /dev/null
+liblockdep.so.*
export Q VERBOSE
-INCLUDES = -I. -I/usr/local/include -I./uinclude -I./include -I../../include $(CONFIG_INCLUDES)
+INCLUDES = -I. -I./uinclude -I./include -I../../include $(CONFIG_INCLUDES)
# Set compile option CFLAGS if not set elsewhere
CFLAGS ?= -g -DCONFIG_LOCKDEP -DCONFIG_STACKTRACE -DCONFIG_PROVE_LOCKING -DBITS_PER_LONG=__WORDSIZE -DLIBLOCKDEP_VERSION='"$(LIBLOCKDEP_VERSION)"' -rdynamic -O0 -g
#include <stdint.h>
#include <limits.h>
+#include <netinet/ip6.h>
#include "event-parse.h"
#include "event-utils.h"
trace_seq_printf(s, fmt, buf[0], buf[1], buf[2], buf[3], buf[4], buf[5]);
}
+static void print_ip4_addr(struct trace_seq *s, char i, unsigned char *buf)
+{
+ const char *fmt;
+
+ if (i == 'i')
+ fmt = "%03d.%03d.%03d.%03d";
+ else
+ fmt = "%d.%d.%d.%d";
+
+ trace_seq_printf(s, fmt, buf[0], buf[1], buf[2], buf[3]);
+}
+
+static inline bool ipv6_addr_v4mapped(const struct in6_addr *a)
+{
+ return ((unsigned long)(a->s6_addr32[0] | a->s6_addr32[1]) |
+ (unsigned long)(a->s6_addr32[2] ^ htonl(0x0000ffff))) == 0UL;
+}
+
+static inline bool ipv6_addr_is_isatap(const struct in6_addr *addr)
+{
+ return (addr->s6_addr32[2] | htonl(0x02000000)) == htonl(0x02005EFE);
+}
+
+static void print_ip6c_addr(struct trace_seq *s, unsigned char *addr)
+{
+ int i, j, range;
+ unsigned char zerolength[8];
+ int longest = 1;
+ int colonpos = -1;
+ uint16_t word;
+ uint8_t hi, lo;
+ bool needcolon = false;
+ bool useIPv4;
+ struct in6_addr in6;
+
+ memcpy(&in6, addr, sizeof(struct in6_addr));
+
+ useIPv4 = ipv6_addr_v4mapped(&in6) || ipv6_addr_is_isatap(&in6);
+
+ memset(zerolength, 0, sizeof(zerolength));
+
+ if (useIPv4)
+ range = 6;
+ else
+ range = 8;
+
+ /* find position of longest 0 run */
+ for (i = 0; i < range; i++) {
+ for (j = i; j < range; j++) {
+ if (in6.s6_addr16[j] != 0)
+ break;
+ zerolength[i]++;
+ }
+ }
+ for (i = 0; i < range; i++) {
+ if (zerolength[i] > longest) {
+ longest = zerolength[i];
+ colonpos = i;
+ }
+ }
+ if (longest == 1) /* don't compress a single 0 */
+ colonpos = -1;
+
+ /* emit address */
+ for (i = 0; i < range; i++) {
+ if (i == colonpos) {
+ if (needcolon || i == 0)
+ trace_seq_printf(s, ":");
+ trace_seq_printf(s, ":");
+ needcolon = false;
+ i += longest - 1;
+ continue;
+ }
+ if (needcolon) {
+ trace_seq_printf(s, ":");
+ needcolon = false;
+ }
+ /* hex u16 without leading 0s */
+ word = ntohs(in6.s6_addr16[i]);
+ hi = word >> 8;
+ lo = word & 0xff;
+ if (hi)
+ trace_seq_printf(s, "%x%02x", hi, lo);
+ else
+ trace_seq_printf(s, "%x", lo);
+
+ needcolon = true;
+ }
+
+ if (useIPv4) {
+ if (needcolon)
+ trace_seq_printf(s, ":");
+ print_ip4_addr(s, 'I', &in6.s6_addr[12]);
+ }
+
+ return;
+}
+
+static void print_ip6_addr(struct trace_seq *s, char i, unsigned char *buf)
+{
+ int j;
+
+ for (j = 0; j < 16; j += 2) {
+ trace_seq_printf(s, "%02x%02x", buf[j], buf[j+1]);
+ if (i == 'I' && j < 14)
+ trace_seq_printf(s, ":");
+ }
+}
+
+/*
+ * %pi4 print an IPv4 address with leading zeros
+ * %pI4 print an IPv4 address without leading zeros
+ * %pi6 print an IPv6 address without colons
+ * %pI6 print an IPv6 address with colons
+ * %pI6c print an IPv6 address in compressed form with colons
+ * %pISpc print an IP address based on sockaddr; p adds port.
+ */
+static int print_ipv4_arg(struct trace_seq *s, const char *ptr, char i,
+ void *data, int size, struct event_format *event,
+ struct print_arg *arg)
+{
+ unsigned char *buf;
+
+ if (arg->type == PRINT_FUNC) {
+ process_defined_func(s, data, size, event, arg);
+ return 0;
+ }
+
+ if (arg->type != PRINT_FIELD) {
+ trace_seq_printf(s, "ARG TYPE NOT FIELD BUT %d", arg->type);
+ return 0;
+ }
+
+ if (!arg->field.field) {
+ arg->field.field =
+ pevent_find_any_field(event, arg->field.name);
+ if (!arg->field.field) {
+ do_warning("%s: field %s not found",
+ __func__, arg->field.name);
+ return 0;
+ }
+ }
+
+ buf = data + arg->field.field->offset;
+
+ if (arg->field.field->size != 4) {
+ trace_seq_printf(s, "INVALIDIPv4");
+ return 0;
+ }
+ print_ip4_addr(s, i, buf);
+
+ return 0;
+}
+
+static int print_ipv6_arg(struct trace_seq *s, const char *ptr, char i,
+ void *data, int size, struct event_format *event,
+ struct print_arg *arg)
+{
+ char have_c = 0;
+ unsigned char *buf;
+ int rc = 0;
+
+ /* pI6c */
+ if (i == 'I' && *ptr == 'c') {
+ have_c = 1;
+ ptr++;
+ rc++;
+ }
+
+ if (arg->type == PRINT_FUNC) {
+ process_defined_func(s, data, size, event, arg);
+ return rc;
+ }
+
+ if (arg->type != PRINT_FIELD) {
+ trace_seq_printf(s, "ARG TYPE NOT FIELD BUT %d", arg->type);
+ return rc;
+ }
+
+ if (!arg->field.field) {
+ arg->field.field =
+ pevent_find_any_field(event, arg->field.name);
+ if (!arg->field.field) {
+ do_warning("%s: field %s not found",
+ __func__, arg->field.name);
+ return rc;
+ }
+ }
+
+ buf = data + arg->field.field->offset;
+
+ if (arg->field.field->size != 16) {
+ trace_seq_printf(s, "INVALIDIPv6");
+ return rc;
+ }
+
+ if (have_c)
+ print_ip6c_addr(s, buf);
+ else
+ print_ip6_addr(s, i, buf);
+
+ return rc;
+}
+
+static int print_ipsa_arg(struct trace_seq *s, const char *ptr, char i,
+ void *data, int size, struct event_format *event,
+ struct print_arg *arg)
+{
+ char have_c = 0, have_p = 0;
+ unsigned char *buf;
+ struct sockaddr_storage *sa;
+ int rc = 0;
+
+ /* pISpc */
+ if (i == 'I') {
+ if (*ptr == 'p') {
+ have_p = 1;
+ ptr++;
+ rc++;
+ }
+ if (*ptr == 'c') {
+ have_c = 1;
+ ptr++;
+ rc++;
+ }
+ }
+
+ if (arg->type == PRINT_FUNC) {
+ process_defined_func(s, data, size, event, arg);
+ return rc;
+ }
+
+ if (arg->type != PRINT_FIELD) {
+ trace_seq_printf(s, "ARG TYPE NOT FIELD BUT %d", arg->type);
+ return rc;
+ }
+
+ if (!arg->field.field) {
+ arg->field.field =
+ pevent_find_any_field(event, arg->field.name);
+ if (!arg->field.field) {
+ do_warning("%s: field %s not found",
+ __func__, arg->field.name);
+ return rc;
+ }
+ }
+
+ sa = (struct sockaddr_storage *) (data + arg->field.field->offset);
+
+ if (sa->ss_family == AF_INET) {
+ struct sockaddr_in *sa4 = (struct sockaddr_in *) sa;
+
+ if (arg->field.field->size < sizeof(struct sockaddr_in)) {
+ trace_seq_printf(s, "INVALIDIPv4");
+ return rc;
+ }
+
+ print_ip4_addr(s, i, (unsigned char *) &sa4->sin_addr);
+ if (have_p)
+ trace_seq_printf(s, ":%d", ntohs(sa4->sin_port));
+
+
+ } else if (sa->ss_family == AF_INET6) {
+ struct sockaddr_in6 *sa6 = (struct sockaddr_in6 *) sa;
+
+ if (arg->field.field->size < sizeof(struct sockaddr_in6)) {
+ trace_seq_printf(s, "INVALIDIPv6");
+ return rc;
+ }
+
+ if (have_p)
+ trace_seq_printf(s, "[");
+
+ buf = (unsigned char *) &sa6->sin6_addr;
+ if (have_c)
+ print_ip6c_addr(s, buf);
+ else
+ print_ip6_addr(s, i, buf);
+
+ if (have_p)
+ trace_seq_printf(s, "]:%d", ntohs(sa6->sin6_port));
+ }
+
+ return rc;
+}
+
+static int print_ip_arg(struct trace_seq *s, const char *ptr,
+ void *data, int size, struct event_format *event,
+ struct print_arg *arg)
+{
+ char i = *ptr; /* 'i' or 'I' */
+ char ver;
+ int rc = 0;
+
+ ptr++;
+ rc++;
+
+ ver = *ptr;
+ ptr++;
+ rc++;
+
+ switch (ver) {
+ case '4':
+ rc += print_ipv4_arg(s, ptr, i, data, size, event, arg);
+ break;
+ case '6':
+ rc += print_ipv6_arg(s, ptr, i, data, size, event, arg);
+ break;
+ case 'S':
+ rc += print_ipsa_arg(s, ptr, i, data, size, event, arg);
+ break;
+ default:
+ return 0;
+ }
+
+ return rc;
+}
+
static int is_printable_array(char *p, unsigned int len)
{
unsigned int i;
ptr++;
arg = arg->next;
break;
+ } else if (*(ptr+1) == 'I' || *(ptr+1) == 'i') {
+ int n;
+
+ n = print_ip_arg(s, ptr+1, data, size, event, arg);
+ if (n > 0) {
+ ptr += n;
+ arg = arg->next;
+ break;
+ }
}
/* fall through */
--remove=::
Remove specified file from the cache.
-M::
---missing=::
+--missing=::
List missing build ids in the cache for the specified file.
-u::
--update::
You should refer to the processor specific documentation for getting these
details. Some of them are referenced in the SEE ALSO section below.
+PARAMETERIZED EVENTS
+--------------------
+
+Some pmu events listed by 'perf-list' will be displayed with '?' in them. For
+example:
+
+ hv_gpci/dtbp_ptitc,phys_processor_idx=?/
+
+This means that when provided as an event, a value for '?' must
+also be supplied. For example:
+
+ perf stat -C 0 -e 'hv_gpci/dtbp_ptitc,phys_processor_idx=0x2/' ...
+
OPTIONS
-------
DESCRIPTION
-----------
-"perf mem -t <TYPE> record" runs a command and gathers memory operation data
+"perf mem record" runs a command and gathers memory operation data
from it, into perf.data. Perf record options are accepted and are passed through.
-"perf mem -t <TYPE> report" displays the result. It invokes perf report with the
-right set of options to display a memory access profile.
+"perf mem report" displays the result. It invokes perf report with the
+right set of options to display a memory access profile. By default, loads
+and stores are sampled. Use the -t option to limit to loads or stores.
Note that on Intel systems the memory latency reported is the use-latency,
not the pure load (or store latency). Use latency includes any pipeline
-t::
--type=::
- Select the memory operation type: load or store (default: load)
+ Select the memory operation type: load or store (default: load,store)
-D::
--dump-raw-samples=::
- a raw PMU event (eventsel+umask) in the form of rNNN where NNN is a
hexadecimal event descriptor.
- - a hardware breakpoint event in the form of '\mem:addr[:access]'
+ - a symbolically formed PMU event like 'pmu/param1=0x3,param2/' where
+ 'param1', 'param2', etc are defined as formats for the PMU in
+ /sys/bus/event_sources/devices/<pmu>/format/*.
+
+ - a symbolically formed event like 'pmu/config=M,config1=N,config3=K/'
+
+ where M, N, K are numbers (in decimal, hex, octal format). Acceptable
+ values for each of 'config', 'config1' and 'config2' are defined by
+ corresponding entries in /sys/bus/event_sources/devices/<pmu>/format/*
+ param1 and param2 are defined as formats for the PMU in:
+ /sys/bus/event_sources/devices/<pmu>/format/*
+
+ - a hardware breakpoint event in the form of '\mem:addr[/len][:access]'
where addr is the address in memory you want to break in.
Access is the memory access type (read, write, execute) it can
- be passed as follows: '\mem:addr[:[r][w][x]]'.
+ be passed as follows: '\mem:addr[:[r][w][x]]'. len is the range,
+ number of bytes from specified addr, which the breakpoint will cover.
If you want to profile read-write accesses in 0x1000, just set
'mem:0x1000:rw'.
+ If you want to profile write accesses in [0x1000~1008), just set
+ 'mem:0x1000/8:w'.
--filter=<filter>::
Event filter.
is equivalent to:
perf script -f trace:<fields> -f sw:<fields> -f hw:<fields>
-
+
i.e., the specified fields apply to all event types if the type string
is not given.
-
+
The arguments are processed in the order received. A later usage can
reset a prior request. e.g.:
-
+
-f trace: -f comm,tid,time,ip,sym
-
+
The first -f suppresses trace events (field list is ""), but then the
second invocation sets the fields to comm,tid,time,ip,sym. In this case a
warning is given to the user:
-
+
"Overriding previous field request for all events."
-
+
Alternatively, consider the order:
-
+
-f comm,tid,time,ip,sym -f trace:
-
+
The first -f sets the fields for all events and the second -f
suppresses trace events. The user is given a warning message about
the override, and the result of the above is that only S/W and H/W
events are displayed with the given fields.
-
+
For the 'wildcard' option if a user selected field is invalid for an
event type, a message is displayed to the user that the option is
ignored for that type. For example:
-
+
$ perf script -f comm,tid,trace
'trace' not valid for hardware events. Ignoring.
'trace' not valid for software events. Ignoring.
-
+
Alternatively, if the type is given an invalid field is specified it
is an error. For example:
-
+
perf script -v -f sw:comm,tid,trace
'trace' not valid for software events.
-
+
At this point usage is displayed, and perf-script exits.
-
+
Finally, a user may not set fields to none for all event types.
i.e., -f "" is not allowed.
-e::
--event=::
- Select the PMU event. Selection can be a symbolic event name
- (use 'perf list' to list all events) or a raw PMU
- event (eventsel+umask) in the form of rNNN where NNN is a
- hexadecimal event descriptor.
+ Select the PMU event. Selection can be:
+
+ - a symbolic event name (use 'perf list' to list all events)
+
+ - a raw PMU event (eventsel+umask) in the form of rNNN where NNN is a
+ hexadecimal event descriptor.
+
+ - a symbolically formed event like 'pmu/param1=0x3,param2/' where
+ param1 and param2 are defined as formats for the PMU in
+ /sys/bus/event_sources/devices/<pmu>/format/*
+
+ - a symbolically formed event like 'pmu/config=M,config1=N,config2=K/'
+ where M, N, K are numbers (in decimal, hex, octal format).
+ Acceptable values for each of 'config', 'config1' and 'config2'
+ parameters are defined by corresponding entries in
+ /sys/bus/event_sources/devices/<pmu>/format/*
-i::
--no-inherit::
val, opflags);
}
+#ifndef HAVE_PTHREAD_ATTR_SETAFFINITY_NP
+#include <pthread.h>
+static inline int pthread_attr_setaffinity_np(pthread_attr_t *attr,
+ size_t cpusetsize,
+ cpu_set_t *cpuset)
+{
+ attr = attr;
+ cpusetsize = cpusetsize;
+ cpuset = cpuset;
+ return 0;
+}
+#endif
+
#endif /* _FUTEX_H */
if (errno == ENOENT)
return false;
- pr_warning("Problems with %s file, consider removing it from the cache\n",
+ pr_warning("Problems with %s file, consider removing it from the cache\n",
filename);
} else if (memcmp(dso->build_id, build_id, sizeof(dso->build_id))) {
- pr_warning("Problems with %s file, consider removing it from the cache\n",
+ pr_warning("Problems with %s file, consider removing it from the cache\n",
filename);
}
}
}
+static struct data__file *fmt_to_data_file(struct perf_hpp_fmt *fmt)
+{
+ struct diff_hpp_fmt *dfmt = container_of(fmt, struct diff_hpp_fmt, fmt);
+ void *ptr = dfmt - dfmt->idx;
+ struct data__file *d = container_of(ptr, struct data__file, fmt);
+
+ return d;
+}
+
static struct hist_entry*
get_pair_data(struct hist_entry *he, struct data__file *d)
{
static struct hist_entry*
get_pair_fmt(struct hist_entry *he, struct diff_hpp_fmt *dfmt)
{
- void *ptr = dfmt - dfmt->idx;
- struct data__file *d = container_of(ptr, struct data__file, fmt);
+ struct data__file *d = fmt_to_data_file(&dfmt->fmt);
return get_pair_data(he, d);
}
next = rb_next(&he->rb_node_in);
if (!hist_entry__next_pair(he)) {
rb_erase(&he->rb_node_in, root);
- hist_entry__free(he);
+ hist_entry__delete(he);
}
}
}
next = rb_first(root);
while (next != NULL) {
struct hist_entry *he, *pair;
+ struct data__file *d;
+ int i;
he = rb_entry(next, struct hist_entry, rb_node_in);
next = rb_next(&he->rb_node_in);
- pair = get_pair_data(he, &data__files[sort_compute]);
- if (!pair)
- continue;
+ data__for_each_file_new(i, d) {
+ pair = get_pair_data(he, d);
+ if (!pair)
+ continue;
- switch (compute) {
- case COMPUTE_DELTA:
- compute_delta(he, pair);
- break;
- case COMPUTE_RATIO:
- compute_ratio(he, pair);
- break;
- case COMPUTE_WEIGHTED_DIFF:
- compute_wdiff(he, pair);
- break;
- default:
- BUG_ON(1);
+ switch (compute) {
+ case COMPUTE_DELTA:
+ compute_delta(he, pair);
+ break;
+ case COMPUTE_RATIO:
+ compute_ratio(he, pair);
+ break;
+ case COMPUTE_WEIGHTED_DIFF:
+ compute_wdiff(he, pair);
+ break;
+ default:
+ BUG_ON(1);
+ }
}
}
}
static int64_t
hist_entry__cmp_compute(struct hist_entry *left, struct hist_entry *right,
- int c)
+ int c, int sort_idx)
{
bool pairs_left = hist_entry__has_pairs(left);
bool pairs_right = hist_entry__has_pairs(right);
if (!pairs_left || !pairs_right)
return pairs_left ? -1 : 1;
- p_left = get_pair_data(left, &data__files[sort_compute]);
- p_right = get_pair_data(right, &data__files[sort_compute]);
+ p_left = get_pair_data(left, &data__files[sort_idx]);
+ p_right = get_pair_data(right, &data__files[sort_idx]);
if (!p_left && !p_right)
return 0;
}
static int64_t
-hist_entry__cmp_nop(struct hist_entry *left __maybe_unused,
+hist_entry__cmp_compute_idx(struct hist_entry *left, struct hist_entry *right,
+ int c, int sort_idx)
+{
+ struct hist_entry *p_right, *p_left;
+
+ p_left = get_pair_data(left, &data__files[sort_idx]);
+ p_right = get_pair_data(right, &data__files[sort_idx]);
+
+ if (!p_left && !p_right)
+ return 0;
+
+ if (!p_left || !p_right)
+ return p_left ? -1 : 1;
+
+ if (c != COMPUTE_DELTA) {
+ /*
+ * The delta can be computed without the baseline, but
+ * others are not. Put those entries which have no
+ * values below.
+ */
+ if (left->dummy && right->dummy)
+ return 0;
+
+ if (left->dummy || right->dummy)
+ return left->dummy ? 1 : -1;
+ }
+
+ return __hist_entry__cmp_compute(p_left, p_right, c);
+}
+
+static int64_t
+hist_entry__cmp_nop(struct perf_hpp_fmt *fmt __maybe_unused,
+ struct hist_entry *left __maybe_unused,
struct hist_entry *right __maybe_unused)
{
return 0;
}
static int64_t
-hist_entry__cmp_baseline(struct hist_entry *left, struct hist_entry *right)
+hist_entry__cmp_baseline(struct perf_hpp_fmt *fmt __maybe_unused,
+ struct hist_entry *left, struct hist_entry *right)
{
- if (sort_compute)
- return 0;
-
if (left->stat.period == right->stat.period)
return 0;
return left->stat.period > right->stat.period ? 1 : -1;
}
static int64_t
-hist_entry__cmp_delta(struct hist_entry *left, struct hist_entry *right)
+hist_entry__cmp_delta(struct perf_hpp_fmt *fmt,
+ struct hist_entry *left, struct hist_entry *right)
{
- return hist_entry__cmp_compute(right, left, COMPUTE_DELTA);
+ struct data__file *d = fmt_to_data_file(fmt);
+
+ return hist_entry__cmp_compute(right, left, COMPUTE_DELTA, d->idx);
}
static int64_t
-hist_entry__cmp_ratio(struct hist_entry *left, struct hist_entry *right)
+hist_entry__cmp_ratio(struct perf_hpp_fmt *fmt,
+ struct hist_entry *left, struct hist_entry *right)
{
- return hist_entry__cmp_compute(right, left, COMPUTE_RATIO);
+ struct data__file *d = fmt_to_data_file(fmt);
+
+ return hist_entry__cmp_compute(right, left, COMPUTE_RATIO, d->idx);
}
static int64_t
-hist_entry__cmp_wdiff(struct hist_entry *left, struct hist_entry *right)
+hist_entry__cmp_wdiff(struct perf_hpp_fmt *fmt,
+ struct hist_entry *left, struct hist_entry *right)
{
- return hist_entry__cmp_compute(right, left, COMPUTE_WEIGHTED_DIFF);
+ struct data__file *d = fmt_to_data_file(fmt);
+
+ return hist_entry__cmp_compute(right, left, COMPUTE_WEIGHTED_DIFF, d->idx);
}
-static void insert_hist_entry_by_compute(struct rb_root *root,
- struct hist_entry *he,
- int c)
+static int64_t
+hist_entry__cmp_delta_idx(struct perf_hpp_fmt *fmt __maybe_unused,
+ struct hist_entry *left, struct hist_entry *right)
{
- struct rb_node **p = &root->rb_node;
- struct rb_node *parent = NULL;
- struct hist_entry *iter;
-
- while (*p != NULL) {
- parent = *p;
- iter = rb_entry(parent, struct hist_entry, rb_node);
- if (hist_entry__cmp_compute(he, iter, c) < 0)
- p = &(*p)->rb_left;
- else
- p = &(*p)->rb_right;
- }
-
- rb_link_node(&he->rb_node, parent, p);
- rb_insert_color(&he->rb_node, root);
+ return hist_entry__cmp_compute_idx(right, left, COMPUTE_DELTA,
+ sort_compute);
}
-static void hists__compute_resort(struct hists *hists)
+static int64_t
+hist_entry__cmp_ratio_idx(struct perf_hpp_fmt *fmt __maybe_unused,
+ struct hist_entry *left, struct hist_entry *right)
{
- struct rb_root *root;
- struct rb_node *next;
-
- if (sort__need_collapse)
- root = &hists->entries_collapsed;
- else
- root = hists->entries_in;
-
- hists->entries = RB_ROOT;
- next = rb_first(root);
-
- hists__reset_stats(hists);
- hists__reset_col_len(hists);
-
- while (next != NULL) {
- struct hist_entry *he;
-
- he = rb_entry(next, struct hist_entry, rb_node_in);
- next = rb_next(&he->rb_node_in);
-
- insert_hist_entry_by_compute(&hists->entries, he, compute);
- hists__inc_stats(hists, he);
+ return hist_entry__cmp_compute_idx(right, left, COMPUTE_RATIO,
+ sort_compute);
+}
- if (!he->filtered)
- hists__calc_col_len(hists, he);
- }
+static int64_t
+hist_entry__cmp_wdiff_idx(struct perf_hpp_fmt *fmt __maybe_unused,
+ struct hist_entry *left, struct hist_entry *right)
+{
+ return hist_entry__cmp_compute_idx(right, left, COMPUTE_WEIGHTED_DIFF,
+ sort_compute);
}
static void hists__process(struct hists *hists)
if (show_baseline_only)
hists__baseline_only(hists);
- if (sort_compute) {
- hists__precompute(hists);
- hists__compute_resort(hists);
- } else {
- hists__output_resort(hists, NULL);
- }
+ hists__precompute(hists);
+ hists__output_resort(hists, NULL);
hists__fprintf(hists, true, 0, 0, 0, stdout);
}
char pfmt[20] = " ";
if (!pair)
- goto dummy_print;
+ goto no_print;
switch (comparison_method) {
case COMPUTE_DELTA:
else
diff = compute_delta(he, pair);
- if (fabs(diff) < 0.01)
- goto dummy_print;
scnprintf(pfmt, 20, "%%%+d.2f%%%%", dfmt->header_width - 1);
return percent_color_snprintf(hpp->buf, hpp->size,
pfmt, diff);
BUG_ON(1);
}
dummy_print:
+ return scnprintf(hpp->buf, hpp->size, "%*s",
+ dfmt->header_width, "N/A");
+no_print:
return scnprintf(hpp->buf, hpp->size, "%*s",
dfmt->header_width, pfmt);
}
else
diff = compute_delta(he, pair);
- if (fabs(diff) >= 0.01)
- scnprintf(buf, size, "%+4.2F%%", diff);
+ scnprintf(buf, size, "%+4.2F%%", diff);
break;
case PERF_HPP_DIFF__RATIO:
/* No point for ratio number if we are dummy.. */
- if (he->dummy)
+ if (he->dummy) {
+ scnprintf(buf, size, "N/A");
break;
+ }
if (pair->diff.computed)
ratio = pair->diff.period_ratio;
case PERF_HPP_DIFF__WEIGHTED_DIFF:
/* No point for wdiff number if we are dummy.. */
- if (he->dummy)
+ if (he->dummy) {
+ scnprintf(buf, size, "N/A");
break;
+ }
if (pair->diff.computed)
wdiff = pair->diff.wdiff;
perf_hpp__register_sort_field(fmt);
}
-static void ui_init(void)
+static int ui_init(void)
{
struct data__file *d;
+ struct perf_hpp_fmt *fmt;
int i;
data__for_each_file(i, d) {
data__hpp_register(d, i ? PERF_HPP_DIFF__PERIOD :
PERF_HPP_DIFF__PERIOD_BASELINE);
}
+
+ if (!sort_compute)
+ return 0;
+
+ /*
+ * Prepend an fmt to sort on columns at 'sort_compute' first.
+ * This fmt is added only to the sort list but not to the
+ * output fields list.
+ *
+ * Note that this column (data) can be compared twice - one
+ * for this 'sort_compute' fmt and another for the normal
+ * diff_hpp_fmt. But it shouldn't a problem as most entries
+ * will be sorted out by first try or baseline and comparing
+ * is not a costly operation.
+ */
+ fmt = zalloc(sizeof(*fmt));
+ if (fmt == NULL) {
+ pr_err("Memory allocation failed\n");
+ return -1;
+ }
+
+ fmt->cmp = hist_entry__cmp_nop;
+ fmt->collapse = hist_entry__cmp_nop;
+
+ switch (compute) {
+ case COMPUTE_DELTA:
+ fmt->sort = hist_entry__cmp_delta_idx;
+ break;
+ case COMPUTE_RATIO:
+ fmt->sort = hist_entry__cmp_ratio_idx;
+ break;
+ case COMPUTE_WEIGHTED_DIFF:
+ fmt->sort = hist_entry__cmp_wdiff_idx;
+ break;
+ default:
+ BUG_ON(1);
+ }
+
+ list_add(&fmt->sort_list, &perf_hpp__sort_list);
+ return 0;
}
static int data_init(int argc, const char **argv)
if (data_init(argc, argv) < 0)
return -1;
- ui_init();
+ if (ui_init() < 0)
+ return -1;
sort__mode = SORT_MODE__DIFF;
int ret = -EINVAL;
struct perf_session *session = inject->session;
struct perf_data_file *file_out = &inject->output;
+ int fd = perf_data_file__fd(file_out);
signal(SIGINT, sig_handler);
}
if (!file_out->is_pipe)
- lseek(file_out->fd, session->header.data_offset, SEEK_SET);
+ lseek(fd, session->header.data_offset, SEEK_SET);
ret = perf_session__process_events(session, &inject->tool);
perf_header__set_feat(&session->header,
HEADER_BUILD_ID);
session->header.data_size = inject->bytes_written;
- perf_session__write_header(session, session->evlist, file_out->fd, true);
+ perf_session__write_header(session, session->evlist, fd, true);
}
return ret;
#include "util/session.h"
#include "util/data.h"
-#define MEM_OPERATION_LOAD "load"
-#define MEM_OPERATION_STORE "store"
-
-static const char *mem_operation = MEM_OPERATION_LOAD;
+#define MEM_OPERATION_LOAD 0x1
+#define MEM_OPERATION_STORE 0x2
struct perf_mem {
struct perf_tool tool;
char const *input_name;
bool hide_unresolved;
bool dump_raw;
+ int operation;
const char *cpu_list;
DECLARE_BITMAP(cpu_bitmap, MAX_NR_CPUS);
};
-static int __cmd_record(int argc, const char **argv)
+static int __cmd_record(int argc, const char **argv, struct perf_mem *mem)
{
int rec_argc, i = 0, j;
const char **rec_argv;
- char event[64];
int ret;
- rec_argc = argc + 4;
+ rec_argc = argc + 7; /* max number of arguments */
rec_argv = calloc(rec_argc + 1, sizeof(char *));
if (!rec_argv)
return -1;
- rec_argv[i++] = strdup("record");
- if (!strcmp(mem_operation, MEM_OPERATION_LOAD))
- rec_argv[i++] = strdup("-W");
- rec_argv[i++] = strdup("-d");
- rec_argv[i++] = strdup("-e");
+ rec_argv[i++] = "record";
- if (strcmp(mem_operation, MEM_OPERATION_LOAD))
- sprintf(event, "cpu/mem-stores/pp");
- else
- sprintf(event, "cpu/mem-loads/pp");
+ if (mem->operation & MEM_OPERATION_LOAD)
+ rec_argv[i++] = "-W";
+
+ rec_argv[i++] = "-d";
+
+ if (mem->operation & MEM_OPERATION_LOAD) {
+ rec_argv[i++] = "-e";
+ rec_argv[i++] = "cpu/mem-loads/pp";
+ }
+
+ if (mem->operation & MEM_OPERATION_STORE) {
+ rec_argv[i++] = "-e";
+ rec_argv[i++] = "cpu/mem-stores/pp";
+ }
- rec_argv[i++] = strdup(event);
for (j = 1; j < argc; j++, i++)
rec_argv[i] = argv[j];
if (!rep_argv)
return -1;
- rep_argv[i++] = strdup("report");
- rep_argv[i++] = strdup("--mem-mode");
- rep_argv[i++] = strdup("-n"); /* display number of samples */
+ rep_argv[i++] = "report";
+ rep_argv[i++] = "--mem-mode";
+ rep_argv[i++] = "-n"; /* display number of samples */
/*
* there is no weight (cost) associated with stores, so don't print
* the column
*/
- if (strcmp(mem_operation, MEM_OPERATION_LOAD))
- rep_argv[i++] = strdup("--sort=mem,sym,dso,symbol_daddr,"
- "dso_daddr,tlb,locked");
+ if (!(mem->operation & MEM_OPERATION_LOAD))
+ rep_argv[i++] = "--sort=mem,sym,dso,symbol_daddr,"
+ "dso_daddr,tlb,locked";
for (j = 1; j < argc; j++, i++)
rep_argv[i] = argv[j];
return ret;
}
+struct mem_mode {
+ const char *name;
+ int mode;
+};
+
+#define MEM_OPT(n, m) \
+ { .name = n, .mode = (m) }
+
+#define MEM_END { .name = NULL }
+
+static const struct mem_mode mem_modes[]={
+ MEM_OPT("load", MEM_OPERATION_LOAD),
+ MEM_OPT("store", MEM_OPERATION_STORE),
+ MEM_END
+};
+
+static int
+parse_mem_ops(const struct option *opt, const char *str, int unset)
+{
+ int *mode = (int *)opt->value;
+ const struct mem_mode *m;
+ char *s, *os = NULL, *p;
+ int ret = -1;
+
+ if (unset)
+ return 0;
+
+ /* str may be NULL in case no arg is passed to -t */
+ if (str) {
+ /* because str is read-only */
+ s = os = strdup(str);
+ if (!s)
+ return -1;
+
+ /* reset mode */
+ *mode = 0;
+
+ for (;;) {
+ p = strchr(s, ',');
+ if (p)
+ *p = '\0';
+
+ for (m = mem_modes; m->name; m++) {
+ if (!strcasecmp(s, m->name))
+ break;
+ }
+ if (!m->name) {
+ fprintf(stderr, "unknown sampling op %s,"
+ " check man page\n", s);
+ goto error;
+ }
+
+ *mode |= m->mode;
+
+ if (!p)
+ break;
+
+ s = p + 1;
+ }
+ }
+ ret = 0;
+
+ if (*mode == 0)
+ *mode = MEM_OPERATION_LOAD;
+error:
+ free(os);
+ return ret;
+}
+
int cmd_mem(int argc, const char **argv, const char *prefix __maybe_unused)
{
struct stat st;
.ordered_events = true,
},
.input_name = "perf.data",
+ /*
+ * default to both load an store sampling
+ */
+ .operation = MEM_OPERATION_LOAD | MEM_OPERATION_STORE,
};
const struct option mem_options[] = {
- OPT_STRING('t', "type", &mem_operation,
- "type", "memory operations(load/store)"),
+ OPT_CALLBACK('t', "type", &mem.operation,
+ "type", "memory operations(load,store) Default load,store",
+ parse_mem_ops),
OPT_BOOLEAN('D', "dump-raw-samples", &mem.dump_raw,
"dump raw samples in ASCII"),
OPT_BOOLEAN('U', "hide-unresolved", &mem.hide_unresolved,
argc = parse_options_subcommand(argc, argv, mem_options, mem_subcommands,
mem_usage, PARSE_OPT_STOP_AT_NON_OPTION);
- if (!argc || !(strncmp(argv[0], "rec", 3) || mem_operation))
+ if (!argc || !(strncmp(argv[0], "rec", 3) || mem.operation))
usage_with_options(mem_usage, mem_options);
if (!mem.input_name || !strlen(mem.input_name)) {
}
if (!strncmp(argv[0], "rec", 3))
- return __cmd_record(argc, argv);
+ return __cmd_record(argc, argv, &mem);
else if (!strncmp(argv[0], "rep", 3))
return report_events(argc, argv, &mem);
else
return rc;
}
+static int process_sample_event(struct perf_tool *tool,
+ union perf_event *event,
+ struct perf_sample *sample,
+ struct perf_evsel *evsel,
+ struct machine *machine)
+{
+ struct record *rec = container_of(tool, struct record, tool);
+
+ rec->samples++;
+
+ return build_id__mark_dso_hit(tool, event, sample, evsel, machine);
+}
+
static int process_buildids(struct record *rec)
{
struct perf_data_file *file = &rec->file;
struct perf_session *session = rec->session;
- u64 start = session->header.data_offset;
- u64 size = lseek(file->fd, 0, SEEK_CUR);
+ u64 size = lseek(perf_data_file__fd(file), 0, SEEK_CUR);
if (size == 0)
return 0;
+ file->size = size;
+
/*
* During this process, it'll load kernel map and replace the
* dso->long_name to a real pathname it found. In this case
*/
symbol_conf.ignore_vmlinux_buildid = true;
- return __perf_session__process_events(session, start,
- size - start,
- size, &build_id__mark_dso_hit_ops);
+ return perf_session__process_events(session, &rec->tool);
}
static void perf_event__synthesize_guest_os(struct machine *machine, void *data)
struct perf_data_file *file = &rec->file;
struct perf_session *session;
bool disabled = false, draining = false;
+ int fd;
rec->progname = argv[0];
return -1;
}
+ fd = perf_data_file__fd(file);
rec->session = session;
record__init_features(rec);
perf_header__clear_feat(&session->header, HEADER_GROUP_DESC);
if (file->is_pipe) {
- err = perf_header__write_pipe(file->fd);
+ err = perf_header__write_pipe(fd);
if (err < 0)
goto out_child;
} else {
- err = perf_session__write_header(session, rec->evlist,
- file->fd, false);
+ err = perf_session__write_header(session, rec->evlist, fd, false);
if (err < 0)
goto out_child;
}
* return this more properly and also
* propagate errors that now are calling die()
*/
- err = perf_event__synthesize_tracing_data(tool, file->fd, rec->evlist,
+ err = perf_event__synthesize_tracing_data(tool, fd, rec->evlist,
process_synthesized_event);
if (err <= 0) {
pr_err("Couldn't record tracing data.\n");
goto out_child;
}
- if (!quiet) {
+ if (!quiet)
fprintf(stderr, "[ perf record: Woken up %ld times to write data ]\n", waking);
- /*
- * Approximate RIP event size: 24 bytes.
- */
- fprintf(stderr,
- "[ perf record: Captured and wrote %.3f MB %s (~%" PRIu64 " samples) ]\n",
- (double)rec->bytes_written / 1024.0 / 1024.0,
- file->path,
- rec->bytes_written / 24);
- }
-
out_child:
if (forks) {
int exit_status;
} else
status = err;
+ /* this will be recalculated during process_buildids() */
+ rec->samples = 0;
+
if (!err && !file->is_pipe) {
rec->session->header.data_size += rec->bytes_written;
if (!rec->no_buildid)
process_buildids(rec);
- perf_session__write_header(rec->session, rec->evlist,
- file->fd, true);
+ perf_session__write_header(rec->session, rec->evlist, fd, true);
+ }
+
+ if (!err && !quiet) {
+ char samples[128];
+
+ if (rec->samples)
+ scnprintf(samples, sizeof(samples),
+ " (%" PRIu64 " samples)", rec->samples);
+ else
+ samples[0] = '\0';
+
+ fprintf(stderr, "[ perf record: Captured and wrote %.3f MB %s%s ]\n",
+ perf_data_file__size(file) / 1024.0 / 1024.0,
+ file->path, samples);
}
out_delete_session:
.default_per_cpu = true,
},
},
+ .tool = {
+ .sample = process_sample_event,
+ .fork = perf_event__process_fork,
+ .comm = perf_event__process_comm,
+ .mmap = perf_event__process_mmap,
+ .mmap2 = perf_event__process_mmap2,
+ },
};
#define CALLCHAIN_HELP "setup and enables call-graph (stack chain/backtrace) recording: "
return perf_default_config(var, value, cb);
}
-static void report__inc_stats(struct report *rep, struct hist_entry *he)
-{
- /*
- * The @he is either of a newly created one or an existing one
- * merging current sample. We only want to count a new one so
- * checking ->nr_events being 1.
- */
- if (he->stat.nr_events == 1)
- rep->nr_entries++;
-}
-
static int hist_iter__report_callback(struct hist_entry_iter *iter,
struct addr_location *al, bool single,
void *arg)
struct mem_info *mi;
struct branch_info *bi;
- report__inc_stats(rep, he);
-
if (!ui__has_annotation())
return 0;
report__warn_kptr_restrict(rep);
+ evlist__for_each(session->evlist, pos)
+ rep->nr_entries += evsel__hists(pos)->nr_entries;
+
if (use_browser == 0) {
if (verbose > 3)
perf_session__fprintf(session, stdout);
"detailed run - start a lot of events"),
OPT_BOOLEAN('S', "sync", &sync_run,
"call sync() before starting a run"),
- OPT_CALLBACK_NOOPT('B', "big-num", NULL, NULL,
+ OPT_CALLBACK_NOOPT('B', "big-num", NULL, NULL,
"print large numbers with thousands\' separators",
stat__set_big_num),
OPT_STRING('C', "cpu", &target.cpu_list, "cpu",
err ? "[unknown]" : uts.release, perf_version_string);
if (use_browser <= 0)
sleep(5);
-
+
map->erange_warned = true;
}
.arg_scnprintf = { [0] = SCA_HEX, /* brk */ }, },
{ .name = "clock_gettime", .errmsg = true, STRARRAY(0, clk_id, clockid), },
{ .name = "close", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_CLOSE_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_CLOSE_FD, /* fd */ }, },
{ .name = "connect", .errmsg = true, },
{ .name = "dup", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "dup2", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "dup3", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "epoll_ctl", .errmsg = true, STRARRAY(1, op, epoll_ctl_ops), },
{ .name = "eventfd2", .errmsg = true,
.arg_scnprintf = { [1] = SCA_EFD_FLAGS, /* flags */ }, },
{ .name = "faccessat", .errmsg = true,
.arg_scnprintf = { [0] = SCA_FDAT, /* dfd */ }, },
{ .name = "fadvise64", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "fallocate", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "fchdir", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "fchmod", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "fchmodat", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FDAT, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FDAT, /* fd */ }, },
{ .name = "fchown", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "fchownat", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FDAT, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FDAT, /* fd */ }, },
{ .name = "fcntl", .errmsg = true,
.arg_scnprintf = { [0] = SCA_FD, /* fd */
[1] = SCA_STRARRAY, /* cmd */ },
.arg_parm = { [1] = &strarray__fcntl_cmds, /* cmd */ }, },
{ .name = "fdatasync", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "flock", .errmsg = true,
.arg_scnprintf = { [0] = SCA_FD, /* fd */
[1] = SCA_FLOCK, /* cmd */ }, },
{ .name = "fsetxattr", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "fstat", .errmsg = true, .alias = "newfstat",
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "fstatat", .errmsg = true, .alias = "newfstatat",
- .arg_scnprintf = { [0] = SCA_FDAT, /* dfd */ }, },
+ .arg_scnprintf = { [0] = SCA_FDAT, /* dfd */ }, },
{ .name = "fstatfs", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "fsync", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "ftruncate", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "futex", .errmsg = true,
.arg_scnprintf = { [1] = SCA_FUTEX_OP, /* op */ }, },
{ .name = "futimesat", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FDAT, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FDAT, /* fd */ }, },
{ .name = "getdents", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "getdents64", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "getitimer", .errmsg = true, STRARRAY(0, which, itimers), },
{ .name = "getrlimit", .errmsg = true, STRARRAY(0, resource, rlimit_resources), },
{ .name = "ioctl", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */
#if defined(__i386__) || defined(__x86_64__)
/*
* FIXME: Make this available to all arches.
{ .name = "kill", .errmsg = true,
.arg_scnprintf = { [1] = SCA_SIGNUM, /* sig */ }, },
{ .name = "linkat", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FDAT, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FDAT, /* fd */ }, },
{ .name = "lseek", .errmsg = true,
.arg_scnprintf = { [0] = SCA_FD, /* fd */
[2] = SCA_STRARRAY, /* whence */ },
.arg_scnprintf = { [0] = SCA_HEX, /* start */
[2] = SCA_MADV_BHV, /* behavior */ }, },
{ .name = "mkdirat", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FDAT, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FDAT, /* fd */ }, },
{ .name = "mknodat", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FDAT, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FDAT, /* fd */ }, },
{ .name = "mlock", .errmsg = true,
.arg_scnprintf = { [0] = SCA_HEX, /* addr */ }, },
{ .name = "mlockall", .errmsg = true,
{ .name = "munmap", .errmsg = true,
.arg_scnprintf = { [0] = SCA_HEX, /* addr */ }, },
{ .name = "name_to_handle_at", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FDAT, /* dfd */ }, },
+ .arg_scnprintf = { [0] = SCA_FDAT, /* dfd */ }, },
{ .name = "newfstatat", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FDAT, /* dfd */ }, },
+ .arg_scnprintf = { [0] = SCA_FDAT, /* dfd */ }, },
{ .name = "open", .errmsg = true,
.arg_scnprintf = { [1] = SCA_OPEN_FLAGS, /* flags */ }, },
{ .name = "open_by_handle_at", .errmsg = true,
{ .name = "poll", .errmsg = true, .timeout = true, },
{ .name = "ppoll", .errmsg = true, .timeout = true, },
{ .name = "pread", .errmsg = true, .alias = "pread64",
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "preadv", .errmsg = true, .alias = "pread",
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "prlimit64", .errmsg = true, STRARRAY(1, resource, rlimit_resources), },
{ .name = "pwrite", .errmsg = true, .alias = "pwrite64",
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "pwritev", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "read", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "readlinkat", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FDAT, /* dfd */ }, },
+ .arg_scnprintf = { [0] = SCA_FDAT, /* dfd */ }, },
{ .name = "readv", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "recvfrom", .errmsg = true,
.arg_scnprintf = { [3] = SCA_MSG_FLAGS, /* flags */ }, },
{ .name = "recvmmsg", .errmsg = true,
{ .name = "recvmsg", .errmsg = true,
.arg_scnprintf = { [2] = SCA_MSG_FLAGS, /* flags */ }, },
{ .name = "renameat", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FDAT, /* dfd */ }, },
+ .arg_scnprintf = { [0] = SCA_FDAT, /* dfd */ }, },
{ .name = "rt_sigaction", .errmsg = true,
.arg_scnprintf = { [0] = SCA_SIGNUM, /* sig */ }, },
{ .name = "rt_sigprocmask", .errmsg = true, STRARRAY(0, how, sighow), },
{ .name = "setitimer", .errmsg = true, STRARRAY(0, which, itimers), },
{ .name = "setrlimit", .errmsg = true, STRARRAY(0, resource, rlimit_resources), },
{ .name = "shutdown", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "socket", .errmsg = true,
.arg_scnprintf = { [0] = SCA_STRARRAY, /* family */
[1] = SCA_SK_TYPE, /* type */ },
.arg_parm = { [0] = &strarray__socket_families, /* family */ }, },
{ .name = "stat", .errmsg = true, .alias = "newstat", },
{ .name = "symlinkat", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FDAT, /* dfd */ }, },
+ .arg_scnprintf = { [0] = SCA_FDAT, /* dfd */ }, },
{ .name = "tgkill", .errmsg = true,
.arg_scnprintf = { [2] = SCA_SIGNUM, /* sig */ }, },
{ .name = "tkill", .errmsg = true,
{ .name = "utimensat", .errmsg = true,
.arg_scnprintf = { [0] = SCA_FDAT, /* dirfd */ }, },
{ .name = "write", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
{ .name = "writev", .errmsg = true,
- .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
+ .arg_scnprintf = { [0] = SCA_FD, /* fd */ }, },
};
static int syscall_fmt__cmp(const void *name, const void *fmtp)
if (thread__priv(thread) == NULL)
thread__set_priv(thread, thread_trace__new());
-
+
if (thread__priv(thread) == NULL)
goto fail;
if (trace->trace_syscalls &&
perf_evlist__add_syscall_newtp(evlist, trace__sys_enter,
trace__sys_exit))
- goto out_error_tp;
+ goto out_error_raw_syscalls;
if (trace->trace_syscalls)
perf_evlist__add_vfs_getname(evlist);
if ((trace->trace_pgfaults & TRACE_PFMAJ) &&
- perf_evlist__add_pgfault(evlist, PERF_COUNT_SW_PAGE_FAULTS_MAJ))
- goto out_error_tp;
+ perf_evlist__add_pgfault(evlist, PERF_COUNT_SW_PAGE_FAULTS_MAJ)) {
+ goto out_error_mem;
+ }
if ((trace->trace_pgfaults & TRACE_PFMIN) &&
perf_evlist__add_pgfault(evlist, PERF_COUNT_SW_PAGE_FAULTS_MIN))
- goto out_error_tp;
+ goto out_error_mem;
if (trace->sched &&
- perf_evlist__add_newtp(evlist, "sched", "sched_stat_runtime",
- trace__sched_stat_runtime))
- goto out_error_tp;
+ perf_evlist__add_newtp(evlist, "sched", "sched_stat_runtime",
+ trace__sched_stat_runtime))
+ goto out_error_sched_stat_runtime;
err = perf_evlist__create_maps(evlist, &trace->opts.target);
if (err < 0) {
{
char errbuf[BUFSIZ];
-out_error_tp:
- perf_evlist__strerror_tp(evlist, errno, errbuf, sizeof(errbuf));
+out_error_sched_stat_runtime:
+ debugfs__strerror_open_tp(errno, errbuf, sizeof(errbuf), "sched", "sched_stat_runtime");
+ goto out_error;
+
+out_error_raw_syscalls:
+ debugfs__strerror_open_tp(errno, errbuf, sizeof(errbuf), "raw_syscalls", "sys_(enter|exit)");
goto out_error;
out_error_mmap:
fprintf(trace->output, "%s\n", errbuf);
goto out_delete_evlist;
}
+out_error_mem:
+ fprintf(trace->output, "Not enough memory to run!\n");
+ goto out_delete_evlist;
}
static int trace__replay(struct trace *trace)
libpython-version \
libslang \
libunwind \
+ pthread-attr-setaffinity-np \
stackprotector-all \
timerfd \
libdw-dwarf-unwind \
libelf-getphdrnum \
libelf-mmap \
libpython-version \
+ pthread-attr-setaffinity-np \
stackprotector-all \
timerfd \
libunwind-debug-frame \
CFLAGS += -DHAVE_SYNC_COMPARE_AND_SWAP_SUPPORT
endif
+ifeq ($(feature-pthread-attr-setaffinity-np), 1)
+ CFLAGS += -DHAVE_PTHREAD_ATTR_SETAFFINITY_NP
+endif
+
ifndef NO_BIONIC
$(call feature_check,bionic)
ifeq ($(feature-bionic), 1)
test-libslang.bin \
test-libunwind.bin \
test-libunwind-debug-frame.bin \
+ test-pthread-attr-setaffinity-np.bin \
test-stackprotector-all.bin \
test-timerfd.bin \
test-libdw-dwarf-unwind.bin \
test-hello.bin:
$(BUILD)
+test-pthread-attr-setaffinity-np.bin:
+ $(BUILD) -Werror -lpthread
+
test-stackprotector-all.bin:
$(BUILD) -Werror -fstack-protector-all
# include "test-zlib.c"
#undef main
+#define main main_test_pthread_attr_setaffinity_np
+# include "test-pthread_attr_setaffinity_np.c"
+#undef main
+
int main(int argc, char *argv[])
{
main_test_libpython();
main_test_libdw_dwarf_unwind();
main_test_sync_compare_and_swap(argc, argv);
main_test_zlib();
+ main_test_pthread_attr_setaffinity_np();
return 0;
}
--- /dev/null
+#include <stdint.h>
+#include <pthread.h>
+
+int main(void)
+{
+ int ret = 0;
+ pthread_attr_t thread_attr;
+
+ pthread_attr_init(&thread_attr);
+ /* don't care abt exact args, just the API itself in libpthread */
+ ret = pthread_attr_setaffinity_np(&thread_attr, 0, NULL);
+
+ return ret;
+}
* ANY CHANGES MADE HERE WILL BE LOST!
*
*/
-
+#include <stdbool.h>
+#ifndef HAS_BOOL
+# define HAS_BOOL 1
+#endif
#line 1 "Context.xs"
/*
* Context.xs. XS interfaces for perf script.
continue
if not self.compare_data(self[t], other[t]):
log.warning("expected %s=%s, got %s" % (t, self[t], other[t]))
-
# Test file description needs to have following sections:
# [config]
he = rb_entry(node, struct hist_entry, rb_node);
rb_erase(node, root_out);
rb_erase(&he->rb_node_in, root_in);
- hist_entry__free(he);
+ hist_entry__delete(he);
}
}
he = rb_entry(node, struct hist_entry, rb_node);
rb_erase(node, root_out);
rb_erase(&he->rb_node_in, root_in);
- hist_entry__free(he);
+ hist_entry__delete(he);
}
}
@cmd="$(PERF)/tests/perf-targz-src-pkg $(PERF)"; \
echo "- $@: $$cmd" && echo $$cmd > $@ && \
( eval $$cmd ) >> $@ 2>&1
-
all: $(run) $(run_O) tarpkg
@echo OK
return 0;
}
+static int test__checkevent_breakpoint_len(struct perf_evlist *evlist)
+{
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
+
+ TEST_ASSERT_VAL("wrong number of entries", 1 == evlist->nr_entries);
+ TEST_ASSERT_VAL("wrong type", PERF_TYPE_BREAKPOINT == evsel->attr.type);
+ TEST_ASSERT_VAL("wrong config", 0 == evsel->attr.config);
+ TEST_ASSERT_VAL("wrong bp_type", (HW_BREAKPOINT_R | HW_BREAKPOINT_W) ==
+ evsel->attr.bp_type);
+ TEST_ASSERT_VAL("wrong bp_len", HW_BREAKPOINT_LEN_1 ==
+ evsel->attr.bp_len);
+
+ return 0;
+}
+
+static int test__checkevent_breakpoint_len_w(struct perf_evlist *evlist)
+{
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
+
+ TEST_ASSERT_VAL("wrong number of entries", 1 == evlist->nr_entries);
+ TEST_ASSERT_VAL("wrong type", PERF_TYPE_BREAKPOINT == evsel->attr.type);
+ TEST_ASSERT_VAL("wrong config", 0 == evsel->attr.config);
+ TEST_ASSERT_VAL("wrong bp_type", HW_BREAKPOINT_W ==
+ evsel->attr.bp_type);
+ TEST_ASSERT_VAL("wrong bp_len", HW_BREAKPOINT_LEN_2 ==
+ evsel->attr.bp_len);
+
+ return 0;
+}
+
+static int
+test__checkevent_breakpoint_len_rw_modifier(struct perf_evlist *evlist)
+{
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
+
+ TEST_ASSERT_VAL("wrong exclude_user", !evsel->attr.exclude_user);
+ TEST_ASSERT_VAL("wrong exclude_kernel", evsel->attr.exclude_kernel);
+ TEST_ASSERT_VAL("wrong exclude_hv", evsel->attr.exclude_hv);
+ TEST_ASSERT_VAL("wrong precise_ip", !evsel->attr.precise_ip);
+
+ return test__checkevent_breakpoint_rw(evlist);
+}
+
static int count_tracepoints(void)
{
char events_path[PATH_MAX];
.check = test__pinned_group,
.id = 41,
},
+ {
+ .name = "mem:0/1",
+ .check = test__checkevent_breakpoint_len,
+ .id = 42,
+ },
+ {
+ .name = "mem:0/2:w",
+ .check = test__checkevent_breakpoint_len_w,
+ .id = 43,
+ },
+ {
+ .name = "mem:0/4:rw:u",
+ .check = test__checkevent_breakpoint_len_rw_modifier,
+ .id = 44
+ },
#if defined(__s390x__)
{
.name = "kvm-s390:kvm_s390_create_vm",
} else {
ret = e->check(evlist);
}
-
+
perf_evlist__delete(evlist);
return ret;
if (type & PERF_SAMPLE_STACK_USER) {
COMP(user_stack.size);
- if (memcmp(s1->user_stack.data, s1->user_stack.data,
+ if (memcmp(s1->user_stack.data, s2->user_stack.data,
s1->user_stack.size)) {
pr_debug("Samples differ at 'user_stack'\n");
return false;
}
annotate_browser__set_top(browser, dl, idx);
-
+
return true;
}
++browser->nr_jumps;
}
-
}
static inline int width_jumps(int n)
}
#define __HPP_SORT_FN(_type, _field) \
-static int64_t hpp__sort_##_type(struct hist_entry *a, struct hist_entry *b) \
+static int64_t hpp__sort_##_type(struct perf_hpp_fmt *fmt __maybe_unused, \
+ struct hist_entry *a, struct hist_entry *b) \
{ \
return __hpp__sort(a, b, he_get_##_field); \
}
}
#define __HPP_SORT_ACC_FN(_type, _field) \
-static int64_t hpp__sort_##_type(struct hist_entry *a, struct hist_entry *b) \
+static int64_t hpp__sort_##_type(struct perf_hpp_fmt *fmt __maybe_unused, \
+ struct hist_entry *a, struct hist_entry *b) \
{ \
return __hpp__sort_acc(a, b, he_get_acc_##_field); \
}
}
#define __HPP_SORT_RAW_FN(_type, _field) \
-static int64_t hpp__sort_##_type(struct hist_entry *a, struct hist_entry *b) \
+static int64_t hpp__sort_##_type(struct perf_hpp_fmt *fmt __maybe_unused, \
+ struct hist_entry *a, struct hist_entry *b) \
{ \
return __hpp__sort(a, b, he_get_raw_##_field); \
}
HPP_RAW_FNS(samples, nr_events)
HPP_RAW_FNS(period, period)
-static int64_t hpp__nop_cmp(struct hist_entry *a __maybe_unused,
+static int64_t hpp__nop_cmp(struct perf_hpp_fmt *fmt __maybe_unused,
+ struct hist_entry *a __maybe_unused,
struct hist_entry *b __maybe_unused)
{
return 0;
#include <linux/types.h>
void ui_progress__finish(void);
-
+
struct ui_progress {
const char *title;
u64 curr, next, step, total;
};
-
+
void ui_progress__init(struct ui_progress *p, u64 total, const char *title);
void ui_progress__update(struct ui_progress *p, u64 adv);
#include "../libslang.h"
char ui_helpline__last_msg[1024];
+bool tui_helpline__set;
static void tui_helpline__pop(void)
{
sizeof(ui_helpline__last_msg) - backlog, format, ap);
backlog += ret;
+ tui_helpline__set = true;
+
if (ui_helpline__last_msg[backlog - 1] == '\n') {
ui_helpline__puts(ui_helpline__last_msg);
SLsmg_refresh();
static volatile int ui__need_resize;
extern struct perf_error_ops perf_tui_eops;
+extern bool tui_helpline__set;
extern void hist_browser__init_hpp(void);
void ui__exit(bool wait_for_ok)
{
- if (wait_for_ok)
+ if (wait_for_ok && tui_helpline__set)
ui__question_window("Fatal Error",
ui_helpline__last_msg,
"Press any key...", 0);
goto out_free_ops;
ops->locked.ins = ins__find(name);
+ free(name);
+
if (ops->locked.ins == NULL)
goto out_free_ops;
if (!ops->locked.ins->ops)
return 0;
- if (ops->locked.ins->ops->parse)
- ops->locked.ins->ops->parse(ops->locked.ops);
+ if (ops->locked.ins->ops->parse &&
+ ops->locked.ins->ops->parse(ops->locked.ops) < 0)
+ goto out_free_ops;
return 0;
static void lock__delete(struct ins_operands *ops)
{
+ struct ins *ins = ops->locked.ins;
+
+ if (ins && ins->ops->free)
+ ins->ops->free(ops->locked.ops);
+ else
+ ins__delete(ops->locked.ops);
+
zfree(&ops->locked.ops);
zfree(&ops->target.raw);
zfree(&ops->target.name);
*s = '\0';
ops->source.raw = strdup(ops->raw);
*s = ',';
-
+
if (ops->source.raw == NULL)
return -1;
if (!dl->ins->ops)
return;
- if (dl->ins->ops->parse)
- dl->ins->ops->parse(&dl->ops);
+ if (dl->ins->ops->parse && dl->ins->ops->parse(&dl->ops) < 0)
+ dl->ins = NULL;
}
static int disasm_line__parse(char *line, char **namep, char **rawp)
int perf_use_color_default = -1;
-static int parse_color(const char *name, int len)
-{
- static const char * const color_names[] = {
- "normal", "black", "red", "green", "yellow",
- "blue", "magenta", "cyan", "white"
- };
- char *end;
- int i;
-
- for (i = 0; i < (int)ARRAY_SIZE(color_names); i++) {
- const char *str = color_names[i];
- if (!strncasecmp(name, str, len) && !str[len])
- return i - 1;
- }
- i = strtol(name, &end, 10);
- if (end - name == len && i >= -1 && i <= 255)
- return i;
- return -2;
-}
-
-static int parse_attr(const char *name, int len)
-{
- static const int attr_values[] = { 1, 2, 4, 5, 7 };
- static const char * const attr_names[] = {
- "bold", "dim", "ul", "blink", "reverse"
- };
- unsigned int i;
-
- for (i = 0; i < ARRAY_SIZE(attr_names); i++) {
- const char *str = attr_names[i];
- if (!strncasecmp(name, str, len) && !str[len])
- return attr_values[i];
- }
- return -1;
-}
-
-void color_parse(const char *value, const char *var, char *dst)
-{
- color_parse_mem(value, strlen(value), var, dst);
-}
-
-void color_parse_mem(const char *value, int value_len, const char *var,
- char *dst)
-{
- const char *ptr = value;
- int len = value_len;
- int attr = -1;
- int fg = -2;
- int bg = -2;
-
- if (!strncasecmp(value, "reset", len)) {
- strcpy(dst, PERF_COLOR_RESET);
- return;
- }
-
- /* [fg [bg]] [attr] */
- while (len > 0) {
- const char *word = ptr;
- int val, wordlen = 0;
-
- while (len > 0 && !isspace(word[wordlen])) {
- wordlen++;
- len--;
- }
-
- ptr = word + wordlen;
- while (len > 0 && isspace(*ptr)) {
- ptr++;
- len--;
- }
-
- val = parse_color(word, wordlen);
- if (val >= -1) {
- if (fg == -2) {
- fg = val;
- continue;
- }
- if (bg == -2) {
- bg = val;
- continue;
- }
- goto bad;
- }
- val = parse_attr(word, wordlen);
- if (val < 0 || attr != -1)
- goto bad;
- attr = val;
- }
-
- if (attr >= 0 || fg >= 0 || bg >= 0) {
- int sep = 0;
-
- *dst++ = '\033';
- *dst++ = '[';
- if (attr >= 0) {
- *dst++ = '0' + attr;
- sep++;
- }
- if (fg >= 0) {
- if (sep++)
- *dst++ = ';';
- if (fg < 8) {
- *dst++ = '3';
- *dst++ = '0' + fg;
- } else {
- dst += sprintf(dst, "38;5;%d", fg);
- }
- }
- if (bg >= 0) {
- if (sep++)
- *dst++ = ';';
- if (bg < 8) {
- *dst++ = '4';
- *dst++ = '0' + bg;
- } else {
- dst += sprintf(dst, "48;5;%d", bg);
- }
- }
- *dst++ = 'm';
- }
- *dst = 0;
- return;
-bad:
- die("bad color value '%.*s' for variable '%s'", value_len, value, var);
-}
-
int perf_config_colorbool(const char *var, const char *value, int stdout_is_tty)
{
if (value) {
int perf_color_default_config(const char *var, const char *value, void *cb);
int perf_config_colorbool(const char *var, const char *value, int stdout_is_tty);
-void color_parse(const char *value, const char *var, char *dst);
-void color_parse_mem(const char *value, int len, const char *var, char *dst);
int color_vsnprintf(char *bf, size_t size, const char *color,
const char *fmt, va_list args);
int color_vfprintf(FILE *fp, const char *color, const char *fmt, va_list args);
break;
cache_offset = offset & DSO__DATA_CACHE_MASK;
- ret = -EINVAL;
- if (-1 == lseek(dso->data.fd, cache_offset, SEEK_SET))
- break;
-
- ret = read(dso->data.fd, cache->data, DSO__DATA_CACHE_SIZE);
+ ret = pread(dso->data.fd, cache->data, DSO__DATA_CACHE_SIZE, cache_offset);
if (ret <= 0)
break;
u32 status_seen;
size_t file_size;
struct list_head open_entry;
+ u64 frame_offset;
} data;
union { /* Tool specific area */
return printed + fprintf(fp, "\n");
}
-int perf_evlist__strerror_tp(struct perf_evlist *evlist __maybe_unused,
- int err, char *buf, size_t size)
-{
- char sbuf[128];
-
- switch (err) {
- case ENOENT:
- scnprintf(buf, size, "%s",
- "Error:\tUnable to find debugfs\n"
- "Hint:\tWas your kernel was compiled with debugfs support?\n"
- "Hint:\tIs the debugfs filesystem mounted?\n"
- "Hint:\tTry 'sudo mount -t debugfs nodev /sys/kernel/debug'");
- break;
- case EACCES:
- scnprintf(buf, size,
- "Error:\tNo permissions to read %s/tracing/events/raw_syscalls\n"
- "Hint:\tTry 'sudo mount -o remount,mode=755 %s'\n",
- debugfs_mountpoint, debugfs_mountpoint);
- break;
- default:
- scnprintf(buf, size, "%s", strerror_r(err, sbuf, sizeof(sbuf)));
- break;
- }
-
- return 0;
-}
-
int perf_evlist__strerror_open(struct perf_evlist *evlist __maybe_unused,
int err, char *buf, size_t size)
{
size_t perf_evlist__fprintf(struct perf_evlist *evlist, FILE *fp);
-int perf_evlist__strerror_tp(struct perf_evlist *evlist, int err, char *buf, size_t size);
int perf_evlist__strerror_open(struct perf_evlist *evlist, int err, char *buf, size_t size);
int perf_evlist__strerror_mmap(struct perf_evlist *evlist, int err, char *buf, size_t size);
if (opts->sample_weight)
perf_evsel__set_sample_bit(evsel, WEIGHT);
+ attr->task = track;
attr->mmap = track;
attr->mmap2 = track && !perf_missing_features.mmap2;
attr->comm = track;
int perf_evsel__alloc_id(struct perf_evsel *evsel, int ncpus, int nthreads)
{
+ if (ncpus == 0 || nthreads == 0)
+ return 0;
+
if (evsel->system_wide)
nthreads = 1;
* - unique number to identify actual perf.data files
* - encode endianness of file
*/
+ ph->version = PERF_HEADER_VERSION_2;
/* check magic number with one endianness */
if (magic == __perf_magic2)
return -1;
ph->needs_swap = true;
- ph->version = PERF_HEADER_VERSION_2;
return 0;
}
return he->stat.period == 0;
}
+static void hists__delete_entry(struct hists *hists, struct hist_entry *he)
+{
+ rb_erase(&he->rb_node, &hists->entries);
+
+ if (sort__need_collapse)
+ rb_erase(&he->rb_node_in, &hists->entries_collapsed);
+
+ --hists->nr_entries;
+ if (!he->filtered)
+ --hists->nr_non_filtered_entries;
+
+ hist_entry__delete(he);
+}
+
void hists__decay_entries(struct hists *hists, bool zap_user, bool zap_kernel)
{
struct rb_node *next = rb_first(&hists->entries);
(zap_kernel && n->level != '.') ||
hists__decay_entry(hists, n)) &&
!n->used) {
- rb_erase(&n->rb_node, &hists->entries);
-
- if (sort__need_collapse)
- rb_erase(&n->rb_node_in, &hists->entries_collapsed);
-
- --hists->nr_entries;
- if (!n->filtered)
- --hists->nr_non_filtered_entries;
-
- hist_entry__free(n);
+ hists__delete_entry(hists, n);
}
}
}
n = rb_entry(next, struct hist_entry, rb_node);
next = rb_next(&n->rb_node);
- rb_erase(&n->rb_node, &hists->entries);
-
- if (sort__need_collapse)
- rb_erase(&n->rb_node_in, &hists->entries_collapsed);
-
- --hists->nr_entries;
- if (!n->filtered)
- --hists->nr_non_filtered_entries;
-
- hist_entry__free(n);
+ hists__delete_entry(hists, n);
}
}
if (!he)
return NULL;
+ hists->nr_entries++;
+
rb_link_node(&he->rb_node_in, parent, p);
rb_insert_color(&he->rb_node_in, hists->entries_in);
out:
if (perf_hpp__should_skip(fmt))
continue;
- cmp = fmt->cmp(left, right);
+ cmp = fmt->cmp(fmt, left, right);
if (cmp)
break;
}
if (perf_hpp__should_skip(fmt))
continue;
- cmp = fmt->collapse(left, right);
+ cmp = fmt->collapse(fmt, left, right);
if (cmp)
break;
}
return cmp;
}
-void hist_entry__free(struct hist_entry *he)
+void hist_entry__delete(struct hist_entry *he)
{
zfree(&he->branch_info);
zfree(&he->mem_info);
iter->callchain,
he->callchain);
}
- hist_entry__free(he);
+ hist_entry__delete(he);
return false;
}
if (perf_hpp__should_skip(fmt))
continue;
- cmp = fmt->sort(a, b);
+ cmp = fmt->sort(fmt, a, b);
if (cmp)
break;
}
int hist_entry__transaction_len(void);
int hist_entry__sort_snprintf(struct hist_entry *he, char *bf, size_t size,
struct hists *hists);
-void hist_entry__free(struct hist_entry *);
+void hist_entry__delete(struct hist_entry *he);
void hists__output_resort(struct hists *hists, struct ui_progress *prog);
void hists__collapse_resort(struct hists *hists, struct ui_progress *prog);
struct hist_entry *he);
int (*entry)(struct perf_hpp_fmt *fmt, struct perf_hpp *hpp,
struct hist_entry *he);
- int64_t (*cmp)(struct hist_entry *a, struct hist_entry *b);
- int64_t (*collapse)(struct hist_entry *a, struct hist_entry *b);
- int64_t (*sort)(struct hist_entry *a, struct hist_entry *b);
+ int64_t (*cmp)(struct perf_hpp_fmt *fmt,
+ struct hist_entry *a, struct hist_entry *b);
+ int64_t (*collapse)(struct perf_hpp_fmt *fmt,
+ struct hist_entry *a, struct hist_entry *b);
+ int64_t (*sort)(struct perf_hpp_fmt *fmt,
+ struct hist_entry *a, struct hist_entry *b);
struct list_head list;
struct list_head sort_list;
#define map__for_each_symbol(map, pos, n) \
dso__for_each_symbol(map->dso, pos, n, map->type)
+/* map__for_each_symbol_with_name - iterate over the symbols in the given map
+ * that have the given name
+ *
+ * @map: the 'struct map *' in which symbols itereated
+ * @sym_name: the symbol name
+ * @pos: the 'struct symbol *' to use as a loop cursor
+ * @filter: to use when loading the DSO
+ */
+#define __map__for_each_symbol_by_name(map, sym_name, pos, filter) \
+ for (pos = map__find_symbol_by_name(map, sym_name, filter); \
+ pos && strcmp(pos->name, sym_name) == 0; \
+ pos = symbol__next_by_name(pos))
+
+#define map__for_each_symbol_by_name(map, sym_name, pos) \
+ __map__for_each_symbol_by_name(map, sym_name, (pos), NULL)
+
typedef int (*symbol_filter_t)(struct map *map, struct symbol *sym);
void map__init(struct map *map, enum map_type type,
}
int parse_events_add_breakpoint(struct list_head *list, int *idx,
- void *ptr, char *type)
+ void *ptr, char *type, u64 len)
{
struct perf_event_attr attr;
if (parse_breakpoint_type(type, &attr))
return -EINVAL;
- /*
- * We should find a nice way to override the access length
- * Provide some defaults for now
- */
- if (attr.bp_type == HW_BREAKPOINT_X)
- attr.bp_len = sizeof(long);
- else
- attr.bp_len = HW_BREAKPOINT_LEN_4;
+ /* Provide some defaults if len is not specified */
+ if (!len) {
+ if (attr.bp_type == HW_BREAKPOINT_X)
+ len = sizeof(long);
+ else
+ len = HW_BREAKPOINT_LEN_4;
+ }
+
+ attr.bp_len = len;
attr.type = PERF_TYPE_BREAKPOINT;
attr.sample_period = 1;
return;
for_each_subsystem(sys_dir, sys_dirent, sys_next) {
- if (subsys_glob != NULL &&
+ if (subsys_glob != NULL &&
!strglobmatch(sys_dirent.d_name, subsys_glob))
continue;
continue;
for_each_event(sys_dirent, evt_dir, evt_dirent, evt_next) {
- if (event_glob != NULL &&
+ if (event_glob != NULL &&
!strglobmatch(evt_dirent.d_name, event_glob))
continue;
for (i = 0; i < max; i++, syms++) {
- if (event_glob != NULL &&
+ if (event_glob != NULL &&
!(strglobmatch(syms->symbol, event_glob) ||
(syms->alias && strglobmatch(syms->alias, event_glob))))
continue;
printf("\n");
printf(" %-50s [%s]\n",
- "mem:<addr>[:access]",
+ "mem:<addr>[/len][:access]",
event_type_descriptors[PERF_TYPE_BREAKPOINT]);
printf("\n");
}
int type_val;
int type_term;
struct list_head list;
+ bool used;
};
struct parse_events_evlist {
int parse_events_add_cache(struct list_head *list, int *idx,
char *type, char *op_result1, char *op_result2);
int parse_events_add_breakpoint(struct list_head *list, int *idx,
- void *ptr, char *type);
+ void *ptr, char *type, u64 len);
int parse_events_add_pmu(struct list_head *list, int *idx,
char *pmu , struct list_head *head_config);
enum perf_pmu_event_symbol_type
<mem>{
{modifier_bp} { return str(yyscanner, PE_MODIFIER_BP); }
: { return ':'; }
+"/" { return '/'; }
{num_dec} { return value(yyscanner, 10); }
{num_hex} { return value(yyscanner, 16); }
/*
}
event_legacy_mem:
+PE_PREFIX_MEM PE_VALUE '/' PE_VALUE ':' PE_MODIFIER_BP sep_dc
+{
+ struct parse_events_evlist *data = _data;
+ struct list_head *list;
+
+ ALLOC_LIST(list);
+ ABORT_ON(parse_events_add_breakpoint(list, &data->idx,
+ (void *) $2, $6, $4));
+ $$ = list;
+}
+|
+PE_PREFIX_MEM PE_VALUE '/' PE_VALUE sep_dc
+{
+ struct parse_events_evlist *data = _data;
+ struct list_head *list;
+
+ ALLOC_LIST(list);
+ ABORT_ON(parse_events_add_breakpoint(list, &data->idx,
+ (void *) $2, NULL, $4));
+ $$ = list;
+}
+|
PE_PREFIX_MEM PE_VALUE ':' PE_MODIFIER_BP sep_dc
{
struct parse_events_evlist *data = _data;
ALLOC_LIST(list);
ABORT_ON(parse_events_add_breakpoint(list, &data->idx,
- (void *) $2, $4));
+ (void *) $2, $4, 0));
$$ = list;
}
|
ALLOC_LIST(list);
ABORT_ON(parse_events_add_breakpoint(list, &data->idx,
- (void *) $2, NULL));
+ (void *) $2, NULL, 0));
$$ = list;
}
return opterror(opt, "is not usable", flags);
if (opt->flags & PARSE_OPT_EXCLUSIVE) {
- if (p->excl_opt) {
+ if (p->excl_opt && p->excl_opt != opt) {
char msg[128];
if (((flags & OPT_SHORT) && p->excl_opt->short_name) ||
}
}
+/*
+ * Term is a string term, and might be a param-term. Try to look up it's value
+ * in the remaining terms.
+ * - We have a term like "base-or-format-term=param-term",
+ * - We need to find the value supplied for "param-term" (with param-term named
+ * in a config string) later on in the term list.
+ */
+static int pmu_resolve_param_term(struct parse_events_term *term,
+ struct list_head *head_terms,
+ __u64 *value)
+{
+ struct parse_events_term *t;
+
+ list_for_each_entry(t, head_terms, list) {
+ if (t->type_val == PARSE_EVENTS__TERM_TYPE_NUM) {
+ if (!strcmp(t->config, term->config)) {
+ t->used = true;
+ *value = t->val.num;
+ return 0;
+ }
+ }
+ }
+
+ if (verbose)
+ printf("Required parameter '%s' not specified\n", term->config);
+
+ return -1;
+}
+
/*
* Setup one of config[12] attr members based on the
* user input data - term parameter.
static int pmu_config_term(struct list_head *formats,
struct perf_event_attr *attr,
struct parse_events_term *term,
+ struct list_head *head_terms,
bool zero)
{
struct perf_pmu_format *format;
__u64 *vp;
+ __u64 val;
+
+ /*
+ * If this is a parameter we've already used for parameterized-eval,
+ * skip it in normal eval.
+ */
+ if (term->used)
+ return 0;
/*
- * Support only for hardcoded and numnerial terms.
* Hardcoded terms should be already in, so nothing
* to be done for them.
*/
if (parse_events__is_hardcoded_term(term))
return 0;
- if (term->type_val != PARSE_EVENTS__TERM_TYPE_NUM)
- return -EINVAL;
-
format = pmu_find_format(formats, term->config);
- if (!format)
+ if (!format) {
+ if (verbose)
+ printf("Invalid event/parameter '%s'\n", term->config);
return -EINVAL;
+ }
switch (format->value) {
case PERF_PMU_FORMAT_VALUE_CONFIG:
}
/*
- * XXX If we ever decide to go with string values for
- * non-hardcoded terms, here's the place to translate
- * them into value.
+ * Either directly use a numeric term, or try to translate string terms
+ * using event parameters.
*/
- pmu_format_value(format->bits, term->val.num, vp, zero);
+ if (term->type_val == PARSE_EVENTS__TERM_TYPE_NUM)
+ val = term->val.num;
+ else if (term->type_val == PARSE_EVENTS__TERM_TYPE_STR) {
+ if (strcmp(term->val.str, "?")) {
+ if (verbose)
+ pr_info("Invalid sysfs entry %s=%s\n",
+ term->config, term->val.str);
+ return -EINVAL;
+ }
+
+ if (pmu_resolve_param_term(term, head_terms, &val))
+ return -EINVAL;
+ } else
+ return -EINVAL;
+
+ pmu_format_value(format->bits, val, vp, zero);
return 0;
}
{
struct parse_events_term *term;
- list_for_each_entry(term, head_terms, list)
- if (pmu_config_term(formats, attr, term, zero))
+ list_for_each_entry(term, head_terms, list) {
+ if (pmu_config_term(formats, attr, term, head_terms, zero))
return -EINVAL;
+ }
return 0;
}
set_bit(b, bits);
}
+static int sub_non_neg(int a, int b)
+{
+ if (b > a)
+ return 0;
+ return a - b;
+}
+
static char *format_alias(char *buf, int len, struct perf_pmu *pmu,
struct perf_pmu_alias *alias)
{
- snprintf(buf, len, "%s/%s/", pmu->name, alias->name);
+ struct parse_events_term *term;
+ int used = snprintf(buf, len, "%s/%s", pmu->name, alias->name);
+
+ list_for_each_entry(term, &alias->terms, list) {
+ if (term->type_val == PARSE_EVENTS__TERM_TYPE_STR)
+ used += snprintf(buf + used, sub_non_neg(len, used),
+ ",%s=%s", term->config,
+ term->val.str);
+ }
+
+ if (sub_non_neg(len, used) > 0) {
+ buf[used] = '/';
+ used++;
+ }
+ if (sub_non_neg(len, used) > 0) {
+ buf[used] = '\0';
+ used++;
+ } else
+ buf[len - 1] = '\0';
+
return buf;
}
}
for (i = 0; i < ntevs; i++) {
- if (tevs[i].point.address) {
+ if (tevs[i].point.address && !tevs[i].point.retprobe) {
tmp = strdup(reloc_sym->name);
if (!tmp)
return -ENOMEM;
return ret;
}
-static char *looking_function_name;
-static int num_matched_functions;
-
-static int probe_function_filter(struct map *map __maybe_unused,
- struct symbol *sym)
+static int find_probe_functions(struct map *map, char *name)
{
- if ((sym->binding == STB_GLOBAL || sym->binding == STB_LOCAL) &&
- strcmp(looking_function_name, sym->name) == 0) {
- num_matched_functions++;
- return 0;
+ int found = 0;
+ struct symbol *sym;
+
+ map__for_each_symbol_by_name(map, name, sym) {
+ if (sym->binding == STB_GLOBAL || sym->binding == STB_LOCAL)
+ found++;
}
- return 1;
+
+ return found;
}
#define strdup_or_goto(str, label) \
struct kmap *kmap = NULL;
struct ref_reloc_sym *reloc_sym = NULL;
struct symbol *sym;
- struct rb_node *nd;
struct probe_trace_event *tev;
struct perf_probe_point *pp = &pev->point;
struct probe_trace_point *tp;
+ int num_matched_functions;
int ret, i;
/* Init maps of given executable or kernel */
* Load matched symbols: Since the different local symbols may have
* same name but different addresses, this lists all the symbols.
*/
- num_matched_functions = 0;
- looking_function_name = pp->function;
- ret = map__load(map, probe_function_filter);
- if (ret || num_matched_functions == 0) {
+ num_matched_functions = find_probe_functions(map, pp->function);
+ if (num_matched_functions == 0) {
pr_err("Failed to find symbol %s in %s\n", pp->function,
target ? : "kernel");
ret = -ENOENT;
goto out;
}
- if (!pev->uprobes) {
+ if (!pev->uprobes && !pp->retprobe) {
kmap = map__kmap(map);
reloc_sym = kmap->ref_reloc_sym;
if (!reloc_sym) {
}
ret = 0;
- map__for_each_symbol(map, sym, nd) {
+
+ map__for_each_symbol_by_name(map, pp->function, sym) {
tev = (*tevs) + ret;
tp = &tev->point;
if (ret == num_matched_functions) {
Py_DECREF(file);
goto free_list;
}
-
+
Py_DECREF(file);
}
/*
* Insert val into into the dictionary and decrement the reference counter.
- * This is necessary for dictionaries since PyDict_SetItemString() does not
+ * This is necessary for dictionaries since PyDict_SetItemString() does not
* steal a reference, as opposed to PyTuple_SetItem().
*/
static void pydict_set_item_string_decref(PyObject *dict, const char *key, PyObject *val)
if (tool->id_index == NULL)
tool->id_index = process_id_index_stub;
}
-
+
static void swap_sample_id_all(union perf_event *event, void *data)
{
void *end = (void *) event + event->header.size;
#define NUM_MMAPS 128
#endif
-int __perf_session__process_events(struct perf_session *session,
- u64 data_offset, u64 data_size,
- u64 file_size, struct perf_tool *tool)
+static int __perf_session__process_events(struct perf_session *session,
+ u64 data_offset, u64 data_size,
+ u64 file_size, struct perf_tool *tool)
{
int fd = perf_data_file__fd(session->file);
u64 head, page_offset, file_offset, file_pos, size;
union perf_event **event_ptr,
struct perf_sample *sample);
-int __perf_session__process_events(struct perf_session *session,
- u64 data_offset, u64 data_size, u64 size,
- struct perf_tool *tool);
int perf_session__process_events(struct perf_session *session,
struct perf_tool *tool);
return hse->se->se_snprintf(he, hpp->buf, hpp->size, len);
}
+static int64_t __sort__hpp_cmp(struct perf_hpp_fmt *fmt,
+ struct hist_entry *a, struct hist_entry *b)
+{
+ struct hpp_sort_entry *hse;
+
+ hse = container_of(fmt, struct hpp_sort_entry, hpp);
+ return hse->se->se_cmp(a, b);
+}
+
+static int64_t __sort__hpp_collapse(struct perf_hpp_fmt *fmt,
+ struct hist_entry *a, struct hist_entry *b)
+{
+ struct hpp_sort_entry *hse;
+ int64_t (*collapse_fn)(struct hist_entry *, struct hist_entry *);
+
+ hse = container_of(fmt, struct hpp_sort_entry, hpp);
+ collapse_fn = hse->se->se_collapse ?: hse->se->se_cmp;
+ return collapse_fn(a, b);
+}
+
+static int64_t __sort__hpp_sort(struct perf_hpp_fmt *fmt,
+ struct hist_entry *a, struct hist_entry *b)
+{
+ struct hpp_sort_entry *hse;
+ int64_t (*sort_fn)(struct hist_entry *, struct hist_entry *);
+
+ hse = container_of(fmt, struct hpp_sort_entry, hpp);
+ sort_fn = hse->se->se_sort ?: hse->se->se_cmp;
+ return sort_fn(a, b);
+}
+
static struct hpp_sort_entry *
__sort_dimension__alloc_hpp(struct sort_dimension *sd)
{
hse->hpp.entry = __sort__hpp_entry;
hse->hpp.color = NULL;
- hse->hpp.cmp = sd->entry->se_cmp;
- hse->hpp.collapse = sd->entry->se_collapse ? : sd->entry->se_cmp;
- hse->hpp.sort = sd->entry->se_sort ? : hse->hpp.collapse;
+ hse->hpp.cmp = __sort__hpp_cmp;
+ hse->hpp.collapse = __sort__hpp_collapse;
+ hse->hpp.sort = __sort__hpp_sort;
INIT_LIST_HEAD(&hse->hpp.list);
INIT_LIST_HEAD(&hse->hpp.sort_list);
const char *ext = strrchr(name, '.');
char tmpbuf[] = "/tmp/perf-kmod-XXXXXX";
- if ((type != DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE_COMP &&
- type != DSO_BINARY_TYPE__GUEST_KMODULE_COMP) ||
- type != dso->symtab_type)
+ if (type != DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE_COMP &&
+ type != DSO_BINARY_TYPE__GUEST_KMODULE_COMP &&
+ type != DSO_BINARY_TYPE__BUILD_ID_CACHE)
return -1;
- if (!ext || !is_supported_compression(ext + 1))
- return -1;
+ if (!ext || !is_supported_compression(ext + 1)) {
+ ext = strrchr(dso->name, '.');
+ if (!ext || !is_supported_compression(ext + 1))
+ return -1;
+ }
fd = mkstemp(tmpbuf);
if (fd < 0)
const char *name)
{
struct rb_node *n;
+ struct symbol_name_rb_node *s;
if (symbols == NULL)
return NULL;
n = symbols->rb_node;
while (n) {
- struct symbol_name_rb_node *s;
int cmp;
s = rb_entry(n, struct symbol_name_rb_node, rb_node);
else if (cmp > 0)
n = n->rb_right;
else
- return &s->sym;
+ break;
}
- return NULL;
+ if (n == NULL)
+ return NULL;
+
+ /* return first symbol that has same name (if any) */
+ for (n = rb_prev(n); n; n = rb_prev(n)) {
+ struct symbol_name_rb_node *tmp;
+
+ tmp = rb_entry(n, struct symbol_name_rb_node, rb_node);
+ if (strcmp(tmp->sym.name, s->sym.name))
+ break;
+
+ s = tmp;
+ }
+
+ return &s->sym;
}
struct symbol *dso__find_symbol(struct dso *dso,
return symbols__next(sym);
}
+struct symbol *symbol__next_by_name(struct symbol *sym)
+{
+ struct symbol_name_rb_node *s = container_of(sym, struct symbol_name_rb_node, sym);
+ struct rb_node *n = rb_next(&s->rb_node);
+
+ return n ? &rb_entry(n, struct symbol_name_rb_node, rb_node)->sym : NULL;
+}
+
+ /*
+ * Teturns first symbol that matched with @name.
+ */
struct symbol *dso__find_symbol_by_name(struct dso *dso, enum map_type type,
const char *name)
{
struct machine *machine = kmaps->machine;
struct map *curr_map = map;
struct symbol *pos;
- int count = 0, moved = 0;
+ int count = 0, moved = 0;
struct rb_root *root = &dso->symbols[map->type];
struct rb_node *next = rb_first(root);
int kernel_range = 0;
u64 addr);
struct symbol *dso__find_symbol_by_name(struct dso *dso, enum map_type type,
const char *name);
+struct symbol *symbol__next_by_name(struct symbol *sym);
struct symbol *dso__first_symbol(struct dso *dso, enum map_type type);
struct symbol *dso__next_symbol(struct symbol *sym);
u64 *fde_count)
{
int ret = -EINVAL, fd;
- u64 offset;
+ u64 offset = dso->data.frame_offset;
- fd = dso__data_fd(dso, machine);
- if (fd < 0)
- return -EINVAL;
+ if (offset == 0) {
+ fd = dso__data_fd(dso, machine);
+ if (fd < 0)
+ return -EINVAL;
- /* Check the .eh_frame section for unwinding info */
- offset = elf_section_offset(fd, ".eh_frame_hdr");
+ /* Check the .eh_frame section for unwinding info */
+ offset = elf_section_offset(fd, ".eh_frame_hdr");
+ dso->data.frame_offset = offset;
+ }
if (offset)
ret = unwind_spec_ehframe(dso, machine, offset,
static int read_unwind_spec_debug_frame(struct dso *dso,
struct machine *machine, u64 *offset)
{
- int fd = dso__data_fd(dso, machine);
+ int fd;
+ u64 ofs = dso->data.frame_offset;
- if (fd < 0)
- return -EINVAL;
+ if (ofs == 0) {
+ fd = dso__data_fd(dso, machine);
+ if (fd < 0)
+ return -EINVAL;
- /* Check the .debug_frame section for unwinding info */
- *offset = elf_section_offset(fd, ".debug_frame");
+ /* Check the .debug_frame section for unwinding info */
+ ofs = elf_section_offset(fd, ".debug_frame");
+ dso->data.frame_offset = ofs;
+ }
+ *offset = ofs;
if (*offset)
return 0;
ncpus=`grep '^processor' /proc/cpuinfo | wc -l`
idlecpus=`mpstat | tail -1 | \
- awk -v ncpus=$ncpus '{ print ncpus * ($7 + $12) / 100 }'`
+ awk -v ncpus=$ncpus '{ print ncpus * ($7 + $NF) / 100 }'`
awk -v ncpus=$ncpus -v idlecpus=$idlecpus < /dev/null '
BEGIN {
cpus2use = idlecpus;
echo Unreadable results directory: $i
exit 1
fi
+. tools/testing/selftests/rcutorture/bin/functions.sh
configfile=`echo $i | sed -e 's/^.*\///'`
ngps=`grep ver: $i/console.log 2> /dev/null | tail -1 | sed -e 's/^.* ver: //' -e 's/ .*$//'`
title="$title ($ngpsps per second)"
fi
echo $title
+ nclosecalls=`grep --binary-files=text 'torture: Reader Batch' $i/console.log | tail -1 | awk '{for (i=NF-8;i<=NF;i++) sum+=$i; } END {print sum}'`
+ if test -z "$nclosecalls"
+ then
+ exit 0
+ fi
+ if test "$nclosecalls" -eq 0
+ then
+ exit 0
+ fi
+ # Compute number of close calls per tenth of an hour
+ nclosecalls10=`awk -v nclosecalls=$nclosecalls -v dur=$dur 'BEGIN { print int(nclosecalls * 36000 / dur) }' < /dev/null`
+ if test $nclosecalls10 -gt 5 -a $nclosecalls -gt 1
+ then
+ print_bug $nclosecalls "Reader Batch close calls in" $(($dur/60)) minute run: $i
+ else
+ print_warning $nclosecalls "Reader Batch close calls in" $(($dur/60)) minute run: $i
+ fi
fi
#
# Usage: kvm-test-1-run.sh config builddir resdir minutes qemu-args boot_args
#
-# qemu-args defaults to "-nographic", along with arguments specifying the
-# number of CPUs and other options generated from
-# the underlying CPU architecture.
+# qemu-args defaults to "-enable-kvm -soundhw pcspk -nographic", along with
+# arguments specifying the number of CPUs and other
+# options generated from the underlying CPU architecture.
# boot_args defaults to value returned by the per_version_boot_params
# shell function.
#
fi
# Generate -smp qemu argument.
-qemu_args="-nographic $qemu_args"
+qemu_args="-enable-kvm -soundhw pcspk -nographic $qemu_args"
cpu_count=`configNR_CPUS.sh $config_template`
cpu_count=`configfrag_boot_cpus "$boot_args" "$config_template" "$cpu_count"`
vcpus=`identify_qemu_vcpus`
touch $resdir/buildonly
exit 0
fi
+echo "NOTE: $QEMU either did not run or was interactive" > $builddir/console.log
echo $QEMU $qemu_args -m 512 -kernel $resdir/bzImage -append \"$qemu_append $boot_args\" > $resdir/qemu-cmd
( $QEMU $qemu_args -m 512 -kernel $resdir/bzImage -append "$qemu_append $boot_args"; echo $? > $resdir/qemu-retval ) &
qemu_pid=$!
#
# Authors: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
-T=$1
+F=$1
title=$2
+T=/tmp/parse-build.sh.$$
+trap 'rm -rf $T' 0
+mkdir $T
. functions.sh
-if grep -q CC < $T
+if grep -q CC < $F
then
:
else
exit 1
fi
-if grep -q "error:" < $T
+if grep -q "error:" < $F
then
print_bug $title build errors:
- grep "error:" < $T
+ grep "error:" < $F
exit 2
fi
-exit 0
-if egrep -q "rcu[^/]*\.c.*warning:|rcu.*\.h.*warning:" < $T
+grep warning: < $F > $T/warnings
+grep "include/linux/*rcu*\.h:" $T/warnings > $T/hwarnings
+grep "kernel/rcu/[^/]*:" $T/warnings > $T/cwarnings
+cat $T/hwarnings $T/cwarnings > $T/rcuwarnings
+if test -s $T/rcuwarnings
then
print_warning $title build errors:
- egrep "rcu[^/]*\.c.*warning:|rcu.*\.h.*warning:" < $T
+ cat $T/rcuwarnings
exit 2
fi
exit 0
then
print_warning Console output contains nul bytes, old qemu still running?
fi
-egrep 'Badness|WARNING:|Warn|BUG|===========|Call Trace:|Oops:' < $file | grep -v 'ODEBUG: ' | grep -v 'Warning: unable to open an initial console' > $T
+egrep 'Badness|WARNING:|Warn|BUG|===========|Call Trace:|Oops:|Stall ended before state dump start' < $file | grep -v 'ODEBUG: ' | grep -v 'Warning: unable to open an initial console' > $T
if test -s $T
then
print_warning Assertion failure in $file $title
projects / karo-tx-linux.git / commitdiff